KR102361025B1 - Wearable glasses and method for displaying image therethrough - Google Patents

Abstract

Wearable glasses according to an embodiment of the present disclosure include a display that displays an image provided by the wearable glasses, a sensing unit that acquires information about a state in which a user is wearing wearable glasses while the image is displayed on the display, and The processor may include a processor that determines a tilt of the wearable glasses for the user by using the wearing state information, and adjusts an image displayed on the display based on the determined tilt.

Description

Wearable glasses and a method of displaying images through wearable glasses {WEARABLE GLASSES AND METHOD FOR DISPLAYING IMAGE THERETHROUGH}

The present disclosure relates to an image display apparatus and method, and more particularly, to wearable glasses and a method for displaying an image on the wearable glasses.

Wearable glasses are a type of head mounted display (HMD) device that can be worn on a user's head to display an image directly in front of the user's eyes as a wearable device worn on the head like glasses.

Since the wearable glasses are temporarily fixed to the user's head by a frame or the like, the position on the user's head may change according to the user's movement. In general, wearable glasses include a near-to-eye display system that displays an image within a few centimeters from a user's eyes. Accordingly, there is a problem in that the output image is greatly distorted and provided to the user due to a minute change in the position of the wearable glasses.

In some embodiments of the present disclosure, by comparing the user's wearing state of the wearable glasses with the reference wearing state, the inclination degree of the wearable glasses is determined with respect to the user, and the output image of the wearable glasses in consideration of the inclination degree of the wearable glasses A system and method for adjusting a device and displaying the adjusted image on a display of wearable glasses are provided.

As a technical means for achieving the above-described technical problem, a first aspect of the present disclosure includes a display that displays an image provided by wearable glasses; a sensing unit configured to obtain information on a state in which a user is wearing the wearable glasses while an image is displayed on the display; and a processor that determines a tilt of the wearable glasses for the user by using the wearing state information, and adjusts an image displayed on the display based on the determined tilt. .

In the wearable glasses according to an embodiment of the present disclosure, the sensing unit acquires the wearing state information including information on a body part of the user who is wearing the wearable glasses, and the processor is configured to: The inclination of the wearable glasses with respect to the user may be determined by comparing the information with the predetermined reference wearing state information.

In the wearable glasses according to an embodiment of the present disclosure, the sensing unit acquires the wearing state information including the image of the body part, and the processor corresponds to the body part from the image of the body part It may be characterized in that the inclination is determined by detecting a region to be used, acquiring a characteristic value from the detected region, and comparing the acquired characteristic value with a reference value included in the reference wearing state information.

In the wearable glasses according to an embodiment of the present disclosure, the wearing state information includes an image obtained by photographing the eyes of the user wearing the wearable glasses, and the processor is configured to extract the eyes from the photographed eye images. obtains at least one of the length of the major axis, the length of the minor axis, the angle of the major axis, the angle of the minor axis, and the position value of the pupil of the It can be characterized by determining.

In the wearable glasses according to an embodiment of the present disclosure, the display displays a test image, and the sensing unit acquires the wearing state information including an image of the user's eyes on which the test image is reflected, The processor detects an area corresponding to the user's eye from the photographed eye image, extracts a reflection image of the test image from within the area corresponding to the eye, and at least one of a size and a shape of the reflection image may be compared with predetermined reference wearing state information, and the inclination may be determined based on the comparison result.

In the wearable glasses according to an embodiment of the present disclosure, the sensing unit may further measure a state value indicating a movement state of the wearable glasses, and when the state value is greater than or equal to a predetermined value, obtain the wearing state information .

In the wearable glasses according to an embodiment of the present disclosure, the processor determines whether the determined inclination is greater than or equal to a predetermined value, and the display is adjusted based on the determined inclination when the determined inclination is less than a predetermined value displayed image, and when the determined inclination is equal to or greater than a predetermined value, a notification image for notifying the user to rewrite the wearable glasses may be displayed.

Meanwhile, a second aspect of the present disclosure may include displaying an image provided by wearable glasses on a display; obtaining information on a state in which a user is wearing the wearable glasses; determining a tilt of the wearable glasses with respect to the user by using the wearing state information; and adjusting the image displayed on the display based on the determined inclination.

In the method of displaying an image by wearable glasses according to an embodiment of the present disclosure, the acquiring of the wearing state information includes information on a body part of the user who is wearing the wearable glasses. obtaining information, wherein the determining of the inclination includes determining the inclination of the wearable glasses for the user by comparing the wearing state information with predetermined reference wearing state information can be done with

In the method for displaying an image by wearable glasses according to an embodiment of the present disclosure, the acquiring of the wearing state information includes acquiring the wearing state information including an image of the body part, The determining of the slope may include: detecting a region corresponding to the body part from the image of the body part; obtaining a characteristic value from the detected region; and determining the inclination by comparing the acquired characteristic value with a reference value included in the reference wearing state information.

In the method of displaying an image by wearable glasses according to an embodiment of the present disclosure, the wearing state information includes an image obtained by photographing the eyes of the user wearing the wearable glasses, and determining the inclination obtaining at least one of a length of a major axis, a length of a minor axis, an angle of a major axis, an angle of a minor axis, and a position value of a pupil of the eye from the photographed eye image; comparing the obtained value with a predetermined reference value; and determining the slope based on the comparison result.

In the method of displaying an image by wearable glasses according to an embodiment of the present disclosure, the displaying of the image includes displaying a test image, and the acquiring of the wearing state information includes: the test image and obtaining the wearing state information including an image of the reflected user's eyes, wherein the determining of the inclination includes detecting a region corresponding to the user's eye from the photographed eye image step; extracting a reflection image of the test image within an area corresponding to the eye; comparing at least one of a size and a shape of the reflected image with predetermined reference wearing state information; and determining the slope based on the comparison result.

In a method for displaying an image by wearable glasses according to an embodiment of the present disclosure, the acquiring of the wearing state information includes: measuring a state value indicating a movement state of the wearable glasses; and when the state value is equal to or greater than a predetermined value, obtaining the wearing state information.

In a method for displaying an image by wearable glasses according to an embodiment of the present disclosure, the displaying of the adjusted image includes: determining whether the determined slope is equal to or greater than a predetermined value; displaying an image for notifying the user to rewrite the wearable glasses when the determined inclination is greater than or equal to a predetermined value; and when the determined inclination is less than a predetermined value, adjusting the image displayed on the display based on the determined inclination.

On the other hand, a third aspect of the present disclosure is a computer-readable recording medium in which one or more programs including instructions for executing a method of displaying an image by wearable glasses are recorded, the method being provided by the wearable glasses displaying an image to obtaining information on a state in which a user is wearing the wearable glasses; determining a tilt of the wearable glasses with respect to the user by using the wearing state information; And it is possible to provide a computer-readable recording medium comprising the step of displaying the adjusted image based on the determined inclination.

On the other hand, a fourth aspect of the present disclosure provides a device for displaying an image through wearable glasses, comprising: a communication unit configured to receive, from the wearable glasses, information about a wearing state in which a user is wearing the wearable glasses, obtained from the wearable glasses; and a controller configured to determine a tilt of the wearable glasses for the user by using the received wearing state information, and adjust an image displayed through the wearable glasses based on the determined tilt, wherein the communication unit comprises: A device may be provided in which an adjusted image is transmitted to the wearable glasses, and the adjusted image is displayed through the wearable glasses.

On the other hand, a fifth aspect of the present disclosure is a method for a device to display an image through wearable glasses, comprising: receiving, from the wearable glasses, information about a wearing state in which a user is wearing the wearable glasses, obtained from the wearable glasses step; determining a tilt of the wearable glasses with respect to the user by using the received wearing state information; and providing an image adjusted based on the determined inclination to the wearable glasses, wherein the adjusted image is displayed through the wearable glasses.

On the other hand, in a sixth aspect of the present disclosure, in the wearable glasses, a display for displaying an image to a user; a sensing unit for obtaining information about the user's gesture; and a processor that determines an adjustment value based on the obtained information and adjusts an image displayed on the display based on the determined adjustment value.

Meanwhile, in a seventh aspect of the present disclosure, there is provided a method for displaying an image by wearable glasses, the method comprising: obtaining information about a user's gesture; determining an adjustment value based on the obtained information; and adjusting the image displayed on the display based on the determined adjustment value.

Meanwhile, an eighth aspect of the present disclosure provides a device for displaying an image through wearable glasses, comprising: a communication unit configured to receive information about a user's gesture obtained from the wearable glasses from the wearable glasses; and a control unit that determines an adjustment value based on the information on the gesture, and adjusts an image displayed through the wearable glasses based on the determined adjustment value, wherein the communication unit transmits the adjusted image to the wearable glasses. provided, and the adjusted image is displayed through the wearable glasses.

1 is a view for explaining the inconvenience felt by a user who uses a general wearable glasses.
2A, 2B, and 2C are diagrams for explaining wearable glasses displaying an image adjusted according to some embodiments of the present disclosure when the wearable glasses are tilted with respect to the user.
3A and 3B illustrate an outer surface of wearable glasses according to some embodiments of the present disclosure;
4A and 4B illustrate an outer surface of wearable glasses according to some embodiments of the present disclosure;
5A and 5B illustrate an outer surface of wearable glasses according to some embodiments of the present disclosure;
6 is a flowchart of a method of displaying an image by wearable glasses according to an embodiment of the present disclosure;
7 is a view for explaining a tilt of wearable glasses for a user determined based on a reference wearing state according to an embodiment of the present disclosure.
8 is a flowchart of a method by which wearable glasses obtain information on a body part of a user according to an embodiment of the present disclosure.
9 is an example of a screen on which a guide image for photographing a user's eyes is displayed, according to an embodiment of the present disclosure.
10A and 10B are examples of images obtained by photographing the user's eyes obtained through the wearable glasses when the wearable glasses are not tilted with respect to the user, according to some embodiments of the present disclosure.
11A, 11B, and 11C are examples of images obtained by photographing the user's eyes obtained through the wearable glasses when the wearable glasses are tilted with respect to the user, according to some embodiments of the present disclosure;
12 is a flowchart of a method for wearable glasses to determine a tilt from an image captured by a user's eyes according to an embodiment of the present disclosure;
13 is a diagram for describing a first area corresponding to an eye determined from an eye image captured according to an embodiment of the present disclosure;
14 is a diagram for describing a second region corresponding to a pupil determined from an eye image captured according to an embodiment of the present disclosure.
15 is a diagram for describing a characteristic value obtained from an eye image captured according to an embodiment of the present disclosure.
16 and 17 are diagrams for explaining a method by which wearable glasses determine a tilt of wearable glasses from an eye image according to some embodiments of the present disclosure;
18 and 19 are flowcharts of a method of displaying an image adjusted based on a tilt of wearable glasses according to an embodiment of the present disclosure.
20, 21A, and 21B are diagrams for explaining that an image displayed through a display is adjusted based on a tilt of wearable glasses according to some embodiments of the present disclosure;
22A, 22B, and 22C are diagrams for explaining a method of adjusting an image displayed through a display based on a tilt of wearable glasses according to some embodiments of the present disclosure;
23 and 24 illustrate examples of animation effects applied to images displayed through wearable glasses according to some embodiments of the present disclosure.
25 is a flowchart of a method for displaying a reduced image of which wearable glasses are adjusted according to an embodiment of the present disclosure;
26 is a view for explaining an image that is reduced and displayed based on a tilt of wearable glasses according to an embodiment of the present disclosure;
27 is a flowchart of a method of displaying an image for notifying a user to rewrite wearable glasses according to an embodiment of the present disclosure;
28 illustrates an example of an image for notifying a user to rewrite wearable glasses according to an embodiment of the present disclosure.
29 illustrates an example of a guide image inducing a user to rewrite wearable glasses according to an embodiment of the present disclosure.
30 is a flowchart of a method of adjusting a display image by wearable glasses according to an embodiment of the present disclosure;
31 is a view for explaining a method of adjusting a position and size of a display image by wearable glasses according to an embodiment of the present disclosure;
32 is a view for explaining a method of adjusting the brightness of a display image by wearable glasses according to an embodiment of the present disclosure;
FIG. 33 is a diagram for explaining a method of setting a reference value in order for wearable glasses to adjust a display image, according to an embodiment of the present disclosure;
34 is a view for explaining a method of correcting a display image by wearable glasses based on a test image reflected from a user's eye according to an embodiment of the present disclosure;
35 is a flowchart of a method of correcting a display image of wearable glasses based on a test image reflected from a user's eye according to some embodiments of the present disclosure;
36A, 36B, and 36C show examples of test images used in accordance with some embodiments of the present disclosure.
37A, 37B, and 37C illustrate examples of images obtained by photographing a user's eyes in which a test image is reflected by wearable glasses according to some embodiments of the present disclosure;
38 is an example of an image obtained by photographing the user's eyes obtained through the wearable glasses when the wearable glasses are not tilted with respect to the user, according to an embodiment of the present disclosure.
39A, 39B, and 39C are examples of images obtained by photographing the user's eyes obtained through the wearable glasses when the wearable glasses are tilted with respect to the user, according to some embodiments of the present disclosure;
40 is a flowchart of a method of determining a tilt of wearable glasses from a photographed eye image according to an embodiment of the present disclosure.
41 is a flowchart of a method of displaying an image adjusted based on a tilt of wearable glasses according to an embodiment of the present disclosure;
42 is a diagram for describing characteristic values of a user's eye measured from an eye image captured according to an embodiment of the present disclosure.
43A, 43B, and 43C are diagrams for explaining a method of determining a gradient based on a reflection image of a test image included in an eye image in which wearable glasses are photographed, according to some embodiments of the present disclosure;
44A and 44B are diagrams for explaining a method of operating wearable glasses based on a captured eye image according to some embodiments of the present disclosure;
45 is a diagram for explaining a method of adjusting a display image based on a user's gesture by wearable glasses according to an embodiment of the present disclosure;
46 is a flowchart of a method by which wearable glasses adjust a display image based on a user's gesture according to an embodiment of the present disclosure;
47 is a flowchart of a method by which wearable glasses capture a user's gesture according to an embodiment of the present disclosure;
48 and 49 are examples of screens on which a guide image for inducing a user's gesture by wearable glasses is displayed, according to some embodiments of the present disclosure.
50 is a flowchart of a method of determining an adjustment value from a gesture image captured by wearable glasses according to an embodiment of the present disclosure;
FIG. 51 is a diagram for describing a method in which wearable glasses detect a user's hand gesture according to an embodiment of the present disclosure;
52, 53, and 54 are diagrams for explaining a method of determining, by wearable glasses, an adjustment value for adjusting a display image based on a user's gesture, according to some embodiments of the present disclosure;
55 and 56 are diagrams for explaining a method by which wearable glasses adjust a display image based on a user's gesture according to some embodiments of the present disclosure;
57 is a diagram illustrating a system in which a device displays an image through wearable glasses according to an embodiment of the present disclosure;
58 is a flowchart of a method for a device to display an image through wearable glasses according to an embodiment of the present disclosure;
59 is a flowchart of a method for a device to display an image through wearable glasses according to an embodiment of the present disclosure;
60 and 61 are block diagrams of wearable glasses according to some embodiments of the present disclosure.
62 and 63 are block diagrams of a device for displaying an image through wearable glasses, according to some embodiments of the present disclosure;

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings so that those of ordinary skill in the art can easily implement them. However, the present invention may be embodied in several different forms and is not limited to the embodiments described herein. And in order to clearly explain the present invention in the drawings, parts irrelevant to the description are omitted, and similar reference numerals are attached to similar parts throughout the specification.

Throughout the specification, when a part "includes" a certain element, it means that other elements may be further included, rather than excluding other elements, unless otherwise stated.

Throughout the specification, the term “display image” refers to an image provided to a user through wearable glasses. The “display image” may include an image previously displayed through the wearable glasses, an image currently being displayed, and an image to be displayed later. The display image may be an image generated by wearable glasses or an image received from an external device.

Hereinafter, the present invention will be described in detail with reference to the accompanying drawings.

The HMD device is a display device that is worn on a user's head to display an image directly in front of the user's eyes. For example, like the wearable glasses 10 shown in FIG. 1 , the HMD device may have the form of glasses. In the present disclosure, various embodiments of the present invention will be described in detail using wearable glasses that are HMD devices in the form of glasses as an example. However, the present invention is not limited to spectacle-type wearable glasses, and the present invention is applicable to various types of HMD devices capable of displaying information within a user's viewing angle by fixing a part of the main body to a user's head. The wearable glasses 10 are temporarily fixed on the user's head by a frame or the like. Accordingly, when the user wears the wearable glasses 10 for a long time, the position of the wearable glasses 10 on the user's head may change according to the user's movement or changes in the external environment.

In general, since the wearable glasses 10 include a near-to-eye display system that displays an image within a few centimeters from the user's eye, the output image is greatly increased by a minute change in the position of the wearable glasses 10 . There is a problem in that it is distorted and provided to the user. That is, when the wearable glasses 10 are tilted or twisted with respect to the user, there is a problem in that the user is provided with a distorted image through the tilted or warped display.

The distorted image may refer to an image provided to a user by changing the size, display position, or shape of the distorted image compared to the reference image. For example, the reference image may be a predetermined image provided to the user in a reference wearing state in which the user is properly wearing the wearable glasses.

Therefore, whenever a user feels discomfort due to a distorted image, there is an inconvenience in that he/she has to take an action to frequently correct the position of the wearable glasses.

The present disclosure recognizes this problem, adjusts a display image based on the inclination of the wearable glasses with respect to the user, and provides a system and method for displaying the adjusted image through the wearable glasses.

2A, 2B, and 2C are diagrams illustrating the degree of inclination of the wearable glasses according to some embodiments of the present disclosure when the wearable glasses are tilted with respect to the user by comparing the wearing state information in which the user is wearing the wearable glasses with the reference wearing state; These are drawings for explaining wearable glasses that display an image adjusted in consideration. The reference wearing state may be a state in which the user is properly wearing the wearable glasses and may be predetermined. Also, the reference wearing state may be set differently for each user who wears the wearable glasses.

The user may receive various information by recognizing the display image provided through the wearable glasses 10 and the surrounding environment together.

FIG. 2A illustrates a case in which the wearable glasses 10 provide a display image to the user in a reference wearing state in which the user is properly wearing the wearable glasses 10 . For example, as shown in FIG. 2A , the wearable glasses 10 may support a navigation function by providing an image 12 guiding a route to a destination.

When the user wears the wearable glasses 10 for a long time, the wearable state of the wearable glasses 10 on the user's head may change according to the user's movement or changes in the external environment. As shown in FIG. 2B , the wearable glasses 10 may be inclined with respect to the user as compared to the reference wearing state shown in FIG. 2A .

When the user wears the general wearable glasses 10 , the user is provided with a distorted image 13 according to the inclination of the wearable glasses 10 . That is, the general wearable glasses 10 outputs the image output by the wearable glasses 10 as it is in the standard wearing state regardless of the inclination of the wearable glasses 10 . For example, as shown in FIG. 2B , when the wearable glasses 10 support a navigation function for guiding a route to a destination, the image 13 provided to the user is visually compared with the actual surrounding environment. It may be displayed tilted. Therefore, according to the prior art, there is an inconvenience in that the user has to physically correct the position of the wearable glasses 10 in order to receive an undistorted image.

Due to this inconvenience, there is a need for a wearable glass 10 capable of outputting a display image by adjusting the display image according to the degree of inclination of the wearable glasses 10 with respect to the user compared to the reference wearing state.

As shown in FIG. 2C , the wearable glasses may be inclined with respect to the user as compared to the reference wearing state shown in FIG. 2A . As shown in FIG. 2C , the wearable glasses 1000 according to an embodiment of the present disclosure may output an adjusted image 14 in which a degree of inclination of the wearable glasses 10 with respect to the user is compensated.

The wearable glasses 1000 according to some embodiments of the present disclosure may determine the degree of inclination of the wearable glasses 1000 based on a predetermined reference wearing state. For example, the reference wearing state may be a state previously determined as a state in which the user is wearing the wearable glasses 1000 correctly. The wearable glasses 1000 according to some embodiments of the present disclosure may output an image adjusted according to the degree of inclination of the wearable glasses 1000 compared to the reference wearing state.

The wearable glasses 1000 according to an embodiment of the present disclosure may provide an image that is not tilted with respect to the user to the user. The wearable glasses 1000 may rotate the display image, move the display position of the display image, enlarge or reduce the size of the display image, or change the shape of the display image, so that the user can compare the wearable state with the reference wearing state. The display image may be adjusted according to the degree to which the wearable glasses 1000 are inclined. For example, as shown in FIG. 2C , when the wearable glasses 1000 support a navigation function for guiding a route to a destination, the wearable glasses 1000 according to an embodiment of the present disclosure may include the wearable glasses 1000 . ) may output the adjusted image 14 according to the degree of inclination. Therefore, the image 14 provided to the user according to an embodiment of the present disclosure is not tilted compared to the actual surrounding environment, that is, information guiding a path to a location matching (matching) the actual surrounding environment. can be provided.

3A and 3B illustrate an outer surface of wearable glasses according to some embodiments of the present disclosure;

3A and 3B , the wearable glasses 1000 according to some embodiments of the present disclosure may be configured in the form of a monocular display.

Referring to FIG. 3A , the wearable glasses 1000 may be configured in the form of glasses fixed to the user's head using the user's ears and nose. However, the configuration of the wearable glasses 1000 is not limited thereto, and, for example, may be attached to a helmet structure or may be deformed in the form of goggles.

The wearable glasses 1000 illustrated in FIG. 3A include a frame 1010 , a processor 1200 , a camera 1050 , an audio output unit 1020 , a display 1030 , a user input unit 1040 , and a power supply unit 1600 . may include However, not all of the components shown in FIG. 3A are essential components of the wearable glasses 1000 . The wearable glasses 1000 may be implemented by more components than those illustrated in FIG. 3A , or the wearable glasses 1000 may be implemented by fewer components than those illustrated in FIG. 3A .

Some components included in the wearable glasses 1000 may be embedded in the wearable glasses 1000 , and some other components may be mounted outside the wearable glasses 1000 . For example, the processor 1200 and the power supply 1600 may be embedded in the wearable glasses 1000 . The display 1030 may be mounted outside the wearable glasses 1000 . Components embedded in the wearable glass 100 and components mounted outside the wearable glass 1000 are not limited thereto.

Components included in the wearable glasses 1000 are not limited to those illustrated in FIG. 3A . For example, although not shown in FIG. 3A , the wearable glasses 1000 may further include a sensing unit (not shown) capable of acquiring information on the wearable glasses 1000 or information on the surrounding environment of the wearable glasses 1000 . and may further include a communication unit (not shown) for communicating with an external device. For example, the sensing unit (not shown) and the communication unit (not shown) may be embedded in the wearable glasses 1000 .

The frame 1010 may be made of a material such as plastic and/or metal, and may further include wires connecting components included in the wearable glass 1000 to each other.

Although the frame illustrated in FIG. 3A is integrally formed, the shape and structure of the frame of the wearable glasses 1000 according to the present disclosure are not limited to those illustrated in FIG. 3A .

For example, the frame 1010 of the wearable glass 1000 may include a connecting member (not shown), so that at least a portion of the frame 1010 may be folded. For example, the wearable glasses 1000 according to an embodiment of the present disclosure include a foldable frame 1010 so that the wearable glasses 1000 can be folded and stored when the user is not using the wearable glasses 1000 . ) can be minimized.

Also, the wearable glasses 1000 according to an embodiment of the present disclosure may further include an elastic band so that the wearable glasses 1000 are fixed to the user's head regardless of the size of the user's head.

Meanwhile, the frame 1010 according to an embodiment of the present disclosure may be configured such that the lens 1005 is detachable. The wearable glasses 1000 according to an embodiment of the present disclosure may not include the lens 1005 . In addition, although the lens 1005 illustrated in FIG. 3A is illustrated as being integrally formed with a nose bridge, the present disclosure is not limited to FIG. 3A . For example, the nose ring of the wearable glass 1000 according to an embodiment of the present disclosure may be formed integrally with the frame 1010 .

The lens 1005 may be made of a transparent material through which a user can see the real space through the lens 1005 . The lens 1005 may be made of a material capable of passing light constituting an image displayed on the display 1030 . Materials that may be used to form the lens 1010 may include, but are not limited to, plastic such as polycarbonate, or glass.

The processor 1200 may be connected to the wearable glasses 1000 by wire or wirelessly. Although the processor 1200 is illustrated as being located on the left side of the frame 1010 in FIG. 3A , the present disclosure is not limited thereto. For example, the processor 1200 may be located on the right side or on the front side of the wearable glasses 1000 adjacent to the camera 1050 .

The processor 1200, for example, receives data from the camera 1050 or the user input units 1040 , analyzes the received data, and through at least one of the display 1030 and the audio output unit 1020 . Information to be transmitted to the user of the wearable glasses 1000 may be generated. The information provided to the user may include at least one of an image, text, video, and audio, but is not limited thereto.

The camera 1050 may be included in the display 1030 or may be located on the frame 1010 as an individual component distinct from the display 1030 . The camera 1050 may be configured as a camera used in a smart phone or a small camera such as a webcam. For example, the camera 1050 may be mounted at a predetermined position suitable for acquiring an image of a user's hand gesture. For example, as shown in FIG. 3A , when the user wears the wearable glasses 1000 , the camera 1050 is mounted at a position adjacent to the user's eye, thereby similar to an image recognized through the user's eye. You can shoot video.

The user input unit 1040 may include at least one of a touch pad operable by a user's finger and a button operable by a user's push operation, but is not limited thereto. Although the user input unit 1040 is illustrated as being located on the side of the frame 1010 in FIG. 3A , it may be located at another position on the wearable glasses 1000 .

The user input unit 1040 receives a user input. The user input may include data or signals input by the user that generate an event synchronous to the start or end of a predetermined operation of the wearable glasses 1000 .

For example, the user input unit 1040 may include an on/off switch capable of turning on/off the power of the wearable glasses 1000 . Alternatively, the user input unit 1040 according to an embodiment of the present disclosure may receive a user input for adjusting an image displayed through the wearable glasses 1000 .

The display 1030 may be located on the upper left side of the lens 1005 as shown in FIG. 3A . As shown in FIG. 3A , the display 1030 may be configured as a translucent optical waveguide (eg, a prism). The display 1030 shown in FIG. 3A reflects the light output from the projector built into the wearable glasses 1000 and displays an image on the fovea of the retina of the user's eye wearing the wearable glasses 1000 . can focus. However, the display 1030 included in the wearable glasses 1000 according to the present disclosure is not limited to FIG. 3A , and an image may be displayed ocularly to the user using various methods and various configurations.

Although not shown in FIG. 3A , the wearable glasses 1000 according to an embodiment of the present disclosure may further include a microphone (not shown). The microphone may receive a user's voice and a sound of an environment surrounding the wearable glasses 1000 .

The audio output unit 1020 may be configured in the form of an earphone that can be mounted on the user's ear of the wearable glasses 1000 . The audio output unit 1020 may be mounted to the wearable glasses 1000 in a fixed form as shown in FIG. 3A , but the present disclosure is not limited thereto. For example, the audio output unit 1020 may be configured to be detachable from the wearable glasses 1000 so that a user of the wearable glasses 1000 can selectively attach the audio output unit 1020 to an ear. For example, the audio output unit 1020 may be configured as a bone conduction speaker.

In FIG. 3A , the power supply 1600 is illustrated as being positioned at an end of the frame 1010 of the wearable glasses 1000 . However, the present disclosure is not limited thereto, and the power supply unit 1600 may be disposed at various positions on the frame 1010 of the wearable glasses 1000 . The power supply unit 1600 supplies power required for the operation of the wearable glasses 1000 to each component. The power supply 1600 may include a battery (not shown) capable of charging power, and may include a cable (not shown) or a cable port (not shown) capable of receiving power from the outside.

3B is an external view of the wearable glasses 1000 as viewed from the direction in which the user looks at the wearable glasses 1000 when the user wears the wearable glasses 1000 .

As shown in FIG. 3B , the wearable glasses 1000 according to an embodiment of the present disclosure may include a camera 1060 facing the user's face adjacent to the display 1030 . The wearable glasses 1000 may adjust an image displayed on the display 1030 based on an image captured by the user's eyes using the camera 1060 . However, the position of the camera 1060 is not limited to the position shown in FIG. 3B , and may be changed to various positions according to the shape of the wearable glasses 1000 .

Also, the wearable glasses 1000 may further include a camera 1070 for eye tracking that faces the user's face. For example, the eye tracking camera 1070 may include an infrared camera. The eye tracking camera 1070 may detect the user's gaze by tracking the user's pupil. However, the position of the eye tracking camera 1070 is not limited to the position shown in FIG. 3B , and may be changed to various positions according to the shape of the wearable glasses 1000 .

Meanwhile, referring to FIGS. 4A and 4B , the wearable glasses 1000 according to some embodiments of the present disclosure have a monocular display shape and may be configured to be fixed to a user's left or right head. With respect to each component included in the wearable glasses 1000 illustrated in FIGS. 4A and 4B , descriptions that overlap with those of FIGS. 3A and 3B will be omitted.

Referring to FIG. 4A , the wearable glasses 1000 may be fixed to the user's head using one side of the user's ear and one side of the nose. The wearable glasses 1000 may be attached to a helmet structure.

The wearable glasses 1000 illustrated in FIG. 4A include a frame 1010 , a processor 1200 , a camera 1050 , an audio output unit 1020 , a display 1030 , a user input unit 1040 , and a power supply unit 1600 . includes

Meanwhile, as shown in FIG. 4A , the wearable glasses 1000 may be configured such that a lens 1005 functions as a display 1030 . In this case, the lens 1005 may be configured as a transparent display or a translucent display. When the lens 1005 is configured as a translucent display, the lens 1005 may include at least one optical waveguide (eg, a prism), an electroluminescent display, or a material such as a liquid crystal display (LCD). may consist of, but is not limited thereto.

Meanwhile, FIG. 4B is an external view of the wearable glasses 1000 viewed from the direction in which the user faces the wearable glasses 1000 when the user wears the wearable glasses 1000 .

As shown in FIG. 4B , the wearable glasses 1000 according to an embodiment of the present disclosure may include a camera 1060 facing the user's face. The wearable glasses 1000 may adjust an image displayed on the display 1030 based on an image captured by the user's eyes using the camera 1060 . However, the position of the camera 1060 is not limited to the position shown in FIG. 4B , and may be changed to various positions according to the shape of the wearable glasses 1000 .

Also, the wearable glasses 1000 may further include a camera 1070 for eye tracking that faces the user's face. The eye tracking camera 1070 may detect the user's gaze by tracking the user's pupil. However, the position of the eye tracking camera 1070 is not limited to the position shown in FIG. 4B , and may be changed to various positions according to the shape of the wearable glasses 1000 .

Meanwhile, FIGS. 5A and 5B illustrate an outer surface of wearable glasses according to some embodiments of the present disclosure.

5A and 5B , the wearable glasses 1000 according to some embodiments of the present disclosure may be configured in the form of a binocular display. With respect to each component included in the wearable glasses 1000 illustrated in FIGS. 5A and 5B , descriptions that overlap with those of FIGS. 3A , 3B , 4A and 4B will be omitted.

The wearable glasses 1000 illustrated in FIG. 5A include a frame 1010 , a processor 1200 , a camera 1050 , an audio output unit 1020 , a display 1030 , a user input unit 1040 , and a power supply unit 1600 . includes

As shown in FIG. 5A , the wearable glasses 1000 may be configured such that a lens 1005 functions as a display 1030 . In this case, the lens 1005 may be configured as a transparent display or a translucent display. When the lens 1005 is configured as a translucent display, the lens 1005 may include at least one optical waveguide (eg, a prism), an electroluminescent display, or a material such as a liquid crystal display (LCD). may consist of, but is not limited thereto.

Meanwhile, FIG. 5B is an external view of the wearable glasses 1000 viewed from the direction in which the user looks at the wearable glasses 1000 when the user wears the wearable glasses 1000 .

As shown in FIG. 5B , the wearable glasses 1000 according to an embodiment of the present disclosure may include a camera 1060 facing the user's face. The wearable glasses 1000 may adjust an image displayed on the display 1030 based on an image captured by the user's eyes using the camera 1060 . However, the position of the camera 1060 is not limited to the position shown in FIG. 5B , and may be changed to various positions according to the shape of the wearable glasses 1000 .

Also, the wearable glasses 1000 may further include a camera 1070 for eye tracking that faces the user's face. The eye tracking camera 1070 may detect the user's gaze by tracking the user's pupil. However, the position of the eye tracking camera 1070 is not limited to the position shown in FIG. 5B , and may be changed to various positions according to the shape of the wearable glasses 1000 . According to an embodiment of the present disclosure, the eye tracking camera 1070 may be omitted, and the function of detecting the user's gaze of the eye tracking camera 1070 by the camera 1060 facing the user's face is performed instead can do.

As described with reference to FIGS. 3A, 3B, 4A, 4B, 5A, and 5B, the shape and components of the wearable glass 1000 according to an embodiment of the present disclosure may be variously modified by those skilled in the art. However, the present disclosure is not limited to FIGS. 3A, 3B, 4A, 4B, 5A, and 5B, and all of the components shown in FIGS. 3A, 3B, 4A, 4B, 5A, and 5B are essential components of the wearable glass 1000 . it is not The wearable glass 1000 may be implemented by more components than the components shown in FIGS. 3A, 3B, 4A, 4B, 5A and 5B, and the components shown in FIGS. 3A, 3B, 4A, 4B, 5A and 5B The wearable glasses 1000 may be implemented with fewer elements than elements.

6 is a flowchart of a method of displaying an image by wearable glasses according to an embodiment of the present disclosure;

In operation S100 , the wearable glasses 1000 according to an embodiment of the present disclosure may display an image provided by the wearable glasses 1000 .

The wearable glasses 1000 may display images processed and provided by the wearable glasses 1000 by outputting them through the display 1030 . The image provided by the wearable glasses 1000 may be an image or a moving image including text, diagram, figure, or color for providing information to the user, and may be a user interface (UI) for interaction between the user and the wearable glasses 1000 . Interface) or an image including a GUI (Graphic User Interface).

Also, for example, the wearable glasses 1000 may display a test image for obtaining information about a wearing state in which the user is wearing the wearable glasses 1000 . The wearable glasses 1000 according to an embodiment of the present disclosure may output a test image, and obtain information about the wearing state in which the user is wearing the wearable glasses 1000 from an image obtained by photographing the user's eyes on which the test image is reflected. have. In step S200 , the wearable glasses 1000 according to an embodiment of the present disclosure may acquire information about a wearing state in which the user is wearing the wearable glasses.

The wearable glasses 1000 according to an embodiment of the present disclosure may determine a state in which the user is wearing the wearable glasses 1000 based on information on a body part of the user wearing the wearable glasses. The wearable glasses 1000 may acquire wearing state information including information on a body part of the user. The information on the body part obtained by the wearable glasses 1000 may include information on the body part on which the wearable glasses 1000 is worn. That is, the information on the body part may include information for determining the relative position and angle of the wearable glasses 1000 with respect to at least one body part included in the head on which the wearable glasses 1000 are worn. For example, the information on the body part may include information on at least one of the eyes, nose, ears, and mouth included in the head of the user who wears the wearable glasses 1000 .

The wearable glasses 1000 may acquire wearing state information including information on a body part of the user that comes into contact with the wearable glasses 1000 . For example, when the wearable glasses 1000 include a nose pad and a temple, the wearable glasses 1000 may include at least one of the position and angle of the nose pad with respect to the user's nose or the position and angle of the temple with respect to the user's ear. Information on at least one of The wearable glasses 1000 may determine whether at least one of the position and angle of the nose ring with respect to the user's nose or the position and angle of the temple with respect to the user's ear has changed, compared with the reference wearing state. Based on the determination result, the HMD device 1000 may acquire information on the state in which the user is currently wearing the wearable glasses 1000 based on the position of the wearable glasses 1000 in the reference wearing state.

Also, the wearable glasses 1000 may acquire wearing state information including information on body parts that are not in contact with the wearable glasses 1000 . For example, the wearable glasses 1000 may photograph the user's eyes, and compare the user's eye image captured in the standard wearing state with the newly photographed eye image. The wearable glasses 1000 compares the user's eye image captured in the standard wearing state with the newly captured eye image, so that the user currently wears the wearable glasses 1000 based on the position of the wearable glasses 1000 in the standard wearing state. It is possible to obtain information about the state of wearing the.

In addition, the wearable glasses 1000 according to an embodiment of the present disclosure have different positions and angles of at least a portion of the wearable glasses 1000 compared to a reference wearing state in which the user properly wears the wearable glasses 1000 . Wearing state information including information indicating whether or not it is lost may be obtained.

The wearable glasses 1000 may obtain information about a wearing state in which the user is wearing the wearable glasses 1000 by using a sensing unit provided in the wearable glasses 1000, or information about the wearing state of the user wearing the wearable glasses 1000 from an external device. can be obtained. For example, the wearable glass 1000 includes a magnetic sensor, an acceleration sensor, a gyroscope sensor, a proximity sensor, an optical sensor, a depth sensor, an infrared sensor, and an ultrasonic sensor provided therein. can be used to obtain wearing state information.

The wearable glasses 1000 according to an embodiment of the present disclosure obtain information about the wearing state in which the user is wearing the wearable glasses 1000 when the wearable glasses 1000 are initialized or when the wearable glasses are turned on. can do.

Alternatively, the wearable glasses 1000 may acquire the wearing state information when a user input for acquiring the wearing state information in which the user is wearing the wearable glasses 1000 is received.

Alternatively, when a large movement of the wearable glasses 1000 occurs, the wearable glasses 1000 may determine that there is a high possibility that the position of the wearable glasses 1000 with respect to the user is changed. Accordingly, when at least one of a measurement value indicating the movement of the wearable glasses 1000 , for example, an acceleration value, a velocity value, or an angular momentum value, is greater than a threshold value, the wearable glasses 1000 may Wearing state information in which the glasses 1000 is worn may be acquired.

The wearable state information in which the user is wearing the wearable glasses 1000 is compared with the reference wearing state in which the user is properly wearing the wearable glasses 1000 to determine the degree of inclination of the wearable glasses 1000 with respect to the user. can be used

The wearable glasses 1000 may obtain wearing state information including information on the body part of the user who is wearing the wearable glasses 1000 in order to determine the degree of inclination of the wearable glasses 1000 with respect to the user. have. As an example, the information on the body part obtained by the wearable glasses 1000 may include information on the user's eyes, nose, ears, mouth, hands, or a combination thereof. Also, the information on the body part obtained by the wearable glasses 1000 may include an image of the body part.

As another example, the information about the user's body part may include information about the user's gesture. A “gesture” may mean a shape represented by a user's body part at a predetermined time point, a change in a shape represented by the body part for a predetermined time, a change in the position of the body part, or an operation of the body part. For example, the information on the gesture may be an image obtained by photographing a body part of the user at a predetermined time point or information about a touch input input through the user input unit 1040 .

As an example, when the user wears the wearable glasses 1000 using a nose bridge as if wearing glasses, the wearable glasses 1000 are the wearable glasses 1000 worn on the user's nose. The degree of inclination may be obtained as information about the user's nose.

As another example, when the user wears the wearable glasses 1000 by fixing the frame of the wearable glasses 1000 to the ears as if wearing glasses, the wearable glasses 1000 may provide the user with the relative position information of both ears of the user. It can be obtained as information about the ears of

As another example, the wearable glasses 1000 may acquire an image of at least one eye of the user as information about the user's eyes. The wearable glasses 1000 may acquire an eye image by photographing the user's eyes using a camera provided in the wearable glasses 1000 . For example, the wearable glasses 1000 may acquire wearing state information including an image obtained by photographing the user's eyes on which the test image is reflected.

A method of acquiring an image obtained by photographing a user's eyes as information on a user's body part will be described later in more detail with reference to FIGS. 8 to 11C .

In operation S300 , the wearable glasses 1000 according to an embodiment of the present disclosure may determine the inclination of the wearable glasses 1000 for the user by using the wearing state information obtained in operation S200 .

The inclination of the wearable glasses 1000 with respect to the user refers to the degree of inclination of the wearable glasses 1000 worn by the user based on a reference wearing state determined in advance as that the user is wearing the wearable glasses 1000 correctly. do. That is, the inclination of the wearable glasses 1000 may mean a change in the current position and angle of the wearable glasses 1000 based on the position and angle of the wearable glasses 1000 with respect to the user in the reference wearing state. The wearable glasses 1000 may determine the inclination of the wearable glasses 1000 for the user by comparing the wearing state information obtained in step S200 with the predetermined reference wearing state information.

For example, the reference wearing state means a state in which the wearable glasses 1000 are worn at a position most suitable for a user to receive an image, and the reference wearing state is stored in advance as a default value or can be set by A method of setting the reference wearing state by the user will be described in detail later with reference to FIG. 33 .

For example, the information about the state in which the user is wearing the wearable glasses 1000 may include an image of the user's body part. The wearable glasses 1000 may detect a region corresponding to the body part from the image of the body part, and obtain a characteristic value of the body part from the detected region. The wearable glasses 1000 may acquire a characteristic value related to a location, shape, or size of a region corresponding to a body part detected in an image of the body part. The wearable glasses 1000 may determine the inclination by comparing the acquired characteristic value with a reference value included in the reference wearing state information. The reference value included in the reference wearing state information may be a characteristic value detected from an image of a body part obtained in the reference wearing state. The reference value may be a value stored in advance or a value set by a user.

The wearable glasses 1000 according to an embodiment of the present disclosure may map and store a plurality of reference wearing states and/or reference values with a plurality of body parts. Alternatively, the wearable glasses 1000 may apply different standards for compensating for the inclination of the wearable glasses 1000 according to users by mapping and storing a plurality of reference wearing states and/or reference values with a plurality of users. have.

A method of determining the inclination of the wearable glasses 1000 from an image of a body part will be described in detail later with reference to FIGS. 12 to 17 .

7 is a view for explaining a tilt of wearable glasses for a user determined based on a reference wearing state according to an embodiment of the present disclosure. The inclination of the wearable glasses 1000 with respect to the user may mean a degree of inclination of the wearable glasses 1000 worn by the user based on a predetermined reference wearing state.

7 illustrates a case in which the wearable glasses 1000 are in a standard wearing state for the user. The reference wearing state refers to a state in which the wearable glasses 1000 are positioned at a position where the user is determined to be most suitable for receiving an image from the wearable glasses 1000, and may be stored in advance as a default value or set by the user. .

For example, the wearable glasses 1000 may obtain an angle at which the wearable glasses 1000 is rotated based on a virtual axis defined in space as a tilt value. For example, the inclination of the wearable glasses 1000 with respect to the user may include a front-to-back inclination, a vertical inclination, and a left-to-right inclination determined based on three axes defined in space in a standard wearing state of the wearable glasses 1000 . . The inclination of the wearable glasses 1000 with respect to the user may mean a relative inclination indicating the degree of inclination of the wearable glasses 1000 with respect to a specific body part of the user based on the reference wearing state.

As shown in FIG. 7 , based on a case in which a state in which the user is wearing the wearable glasses 1000 is the reference wearing state, an x-axis, a y-axis, and a z-axis may be defined centered on the wearable glasses 1000 . have. In the reference wearing state, the x-axis of the wearable glasses 1000 may be parallel to an axis where a coronal plane and a transverse plane of the user's body intersect. In the reference wearing state, the y-axis of the wearable glasses 1000 may be parallel to an axis where a sagittal plane and a transverse plane of the user's body intersect. In the reference wearing state, the z-axis of the wearable glasses 1000 may be parallel to an axis where a sagittal plane and a coronal plane of the user's body intersect. However, the standard wearing state of the present invention is not limited to the above, and can be changed according to the user's setting,

In the present specification, the front and rear inclination of the wearable glasses 1000 refers to an angle rotated about the x-axis based on the position of the wearable glasses 1000 with respect to the user in the reference wearing state (hereinafter referred to as a reference position). For example, the x-axis may be an axis parallel to a straight line connecting both eyes of the user in the standard wearing state.

The vertical inclination of the wearable glass 1000 means an angle at which the wearable glass 1000 is rotated from the reference position about the y-axis. For example, the y-axis may be an axis parallel to the gaze direction in which the user faces the front in the standard wearing state.

The left and right inclination of the wearable glass 1000 means an angle at which the wearable glass 1000 is rotated from the reference position about the z-axis. The z-axis may be an axis perpendicular to the x-axis and the y-axis.

In the present specification, for convenience of explanation, a case in which the rotation angle of the wearable glass 1000 with respect to the x-axis, y-axis, and z-axis illustrated in FIG. 7 is determined as the inclination of the wearable glass 1000 will be described as an example. However, the present disclosure is not limited to FIG. 7 , and various methods and various criteria may be used in determining the degree of inclination of the wearable glasses 1000 with respect to the user.

A method of determining the inclination of the wearable glasses 1000 for the user by using the user's wearing state information will be described later in more detail with reference to FIGS. 12 to 17 .

Meanwhile, the wearable glasses 1000 according to an embodiment of the present disclosure may determine the inclination of the wearable glasses 1000 based on information about the user's gesture. The wearable glasses 1000 may map and store information on various gestures and gradients, or store a relational expression capable of calculating gradients based on information on gestures. The wearable glasses 1000 may determine the inclination by searching for a pre-stored gradient based on the obtained information or calculating the gradient using the searched relational expression.

In operation S400 , the wearable glasses 1000 according to an embodiment of the present disclosure adjust an image displayed on the display 1030 based on the determined inclination. The wearable glasses 1000 adjust the image displayed on the display 1030 based on the inclination determined based on the reference wearing state.

The wearable glasses 1000 according to an embodiment of the present disclosure adjust the image displayed through the display 1030 based on the inclination of the wearable glasses 10 with respect to the user, thereby tilting the wearable glasses 1000 . In spite of this, the user may be provided with an undistorted image.

For example, the wearable glasses 1000 may adjust at least one of a rotation angle, a size, and a shape of the display image based on the inclination of the wearable glasses 1000 determined in step S300 .

Also, for example, when the display 1030 of the wearable glasses 1000 is configured as a translucent light wavelength guide (eg, a prism), the wearable glasses 1000 display the display based on the inclination of the wearable glasses 1000 . It is possible to adjust the position of the focal point where the light output from the projector of 1030 is focused.

The wearable glasses 1000 according to an embodiment of the present disclosure include image adjustments (eg, horizontal translation, vertical translation), kissing necessary to compensate for the inclination of the wearable glasses 1000 . Toning (keystoning) and/or various image processing for correcting a user to receive an undistorted image) may be performed.

For example, when the wearable glasses 1000 are tilted up and down with respect to the user, that is, when the vertical tilt is determined to be a value other than 0, the vertical tilt of the wearable glasses 1000 may be compensated for the display image. That is, the displayed image may be rotated in the opposite direction in which the wearable glasses are inclined by the determined vertical inclination.

Accordingly, a user who uses the wearable glasses 1000 according to an embodiment of the present disclosure may determine the position of the wearable glasses 1000 even when the state in which the user is wearing the wearable glasses 1000 deviates from the reference wearing state. Even if you do not take an action to correct it, you can receive an image in which the tilt is compensated.

A method of displaying an image adjusted based on the inclination of the wearable glasses 1000 will be described later in more detail with reference to FIGS. 19 to 26 .

8 is a flowchart of a method by which wearable glasses obtain information on a body part of a user according to an embodiment of the present disclosure.

When the wearable state information in which the user is wearing the wearable glasses 1000 is used to determine the degree of inclination of the wearable glasses 1000 based on the reference wearing state, the wearable glasses 1000 according to an embodiment of the present disclosure may acquire the user's wearing state information at predetermined time intervals. For example, the wearable glasses 1000 may perform an operation of photographing the user's eyes at predetermined time intervals in order to obtain the user's wearing state information.

However, continuously acquiring the user's wearing state information may result in overload of the memory capacity and the amount of computation of the processor. Accordingly, according to an embodiment of the present disclosure, as shown in FIG. 8 , information on the user's wearing state may be acquired only when a predetermined event occurs.

In operation S210, the wearable glasses 1000 according to an embodiment of the present disclosure determine whether the wearable glasses 1000 are initialized. For example, the wearable glasses 1000 may be initialized when power is turned on or when an initialization input for initializing the wearable glasses 1000 is received from a user.

The wearable glasses 1000 according to an embodiment of the present disclosure may obtain information on the user's wearing state when the wearable glasses 1000 are initialized ( S250 ). After the wearable glasses 1000 are initialized, the wearable glasses 1000 may obtain information on the user's wearing state based on the movement state value by performing step S220 .

In operation S220 , the wearable glasses 1000 according to an embodiment of the present disclosure measure a motion state value of the wearable glasses 1000 . The motion state value means a value representing the motion of the wearable glasses 1000 .

For example, the wearable glasses 1000 may measure at least one of an acceleration value, a velocity value, and an angular momentum value as a motion state value. The wearable glasses 1000 may determine whether the movement state value is greater than or equal to a predetermined value (S230), and when the measured movement state value is greater than or equal to a predetermined value, may acquire wearing state information (S250). For example, when the motion state value of the wearable glasses 1000 is less than a predetermined value, the wearable glasses 1000 may photograph the user's eyes based on the user input by performing step S240 .

In step S240 , the wearable glasses 1000 according to an embodiment of the present disclosure determine whether an input for adjusting a display image is received from a user. When the position of the wearable glasses 1000 is changed and a distorted screen is provided, the user causes the wearable glasses 1000 to adjust the display image based on the degree of inclination of the wearable glasses 1000 based on the reference wearing state. You can enter commands.

The wearable glasses 1000 determine whether a user input for adjusting an image is received ( S240 ). When a user input for adjusting an image is received, wearing state information may be acquired (S250). When the user input of the wearable glasses 1000 is not received, steps S210 to S240 may be repeatedly performed.

In operation S250 , the wearable glasses 1000 according to an embodiment of the present disclosure may acquire information about the user's wearing state. The wearable glasses 1000 may increase the accuracy of measuring the tilt of the wearable glasses 1000 by inducing the user to look at the front by outputting a guide image. For example, as shown in FIG. 9 , the wearable glasses 1000 may display an image 901 notifying the user that the user's eyes are photographed for image adjustment on the display 1030 . The wearable glasses 1000 according to an embodiment of the present disclosure may photograph the user's eyes after outputting a guide image. However, the present disclosure is not limited thereto, and the user's eyes may be photographed without outputting a guide image.

The wearable glasses 1000 according to an embodiment of the present disclosure may photograph a body part of a user in order to obtain information about a state in which the user is wearing the wearable glasses 1000 . For example, the wearable glasses 1000 may obtain wearing state information including an image obtained by photographing the eyes of a user wearing the wearable glasses 1000 . 10A, 10B, 11A, 11B, and 11C are examples of images obtained by photographing the user's eyes obtained through the wearable glasses 1000, according to some embodiments of the present disclosure.

As shown in FIG. 10A , the wearable glasses 1000 may capture a user's monocular image. Alternatively, as shown in FIG. 10B , the wearable glasses 1000 may photograph both eyes of the user.

10A and 10B illustrate images of the user's eyes obtained through the wearable glasses 1000 according to some embodiments of the present disclosure when the wearable glasses 1000 are not tilted with respect to the user based on the reference wearing state. This is an example of a video.

Meanwhile, FIGS. 11A, 11B and 11C illustrate the user's eyes obtained through the wearable glasses 1000 according to some embodiments of the present disclosure when the wearable glasses 1000 are tilted with respect to the user based on the reference wearing state. This is an example of a recorded video.

11A illustrates an example of an image obtained by photographing the user's eyes obtained through the wearable glasses 1000 when the wearable glasses 1000 are tilted up and down with respect to the user and there is a vertical inclination. 11A is a diagram illustrating the user's actions when the wearable glasses 1000 are rotated with respect to the y-axis of the wearable glasses 1000 in the standard wearing state (ie, an axis parallel to the direction in which the user faces the front in the standard wearing state). An image of the eye is shown. It can be seen that the positions and angles of both eyes shown in FIG. 11A are changed compared with the positions of both eyes shown in FIG. 10B .

11B illustrates an example of an image obtained by photographing the user's eyes obtained through the wearable glasses 1000 when the wearable glasses 1000 are tilted left and right with respect to the user and there is a left and right inclination. 11B shows the wearable glasses (ie, the x-axis and the y-axis, which are axes parallel to the straight line connecting the user's eyes in the standard wear state) of the wearable glasses 1000 in the standard wearing state, based on the z-axis ( 1000) shows an image captured by the user's eyes when rotated.

When the distance from one side of the wearable glasses 1000 to the user's predetermined eye increases due to the wearable glasses 1000 being tilted left and right with respect to the user, the size of the eyes shown in the image obtained by photographing the corresponding eyes decreases. In addition, when the distance from the other side of the wearable glasses 1000 to the user's predetermined eye gets closer as the wearable glasses 1000 are tilted left and right with respect to the user, the size of the eyes shown in the image obtained by photographing the corresponding eyes increases. . It can be seen that the size of the eyes shown in FIG. 11B has changed compared with the positions of both eyes shown in FIG. 10A . Referring to FIG. 11B , it can be seen that the distance from the right side of the wearable glasses 1000 to the user's right eye is increased because the user's right eye displayed on the photographed eye image is reduced. Also, since the user's left eye appearing on the photographed eye image has increased, it can be seen that the distance from the left side of the wearable glasses 1000 to the user's left eye has increased. That is, in FIG. 11B , the wearable glass 1000 rotates in a direction where the right end of the wearable glass 1000 moves away from the user and the left end approaches the user with respect to the z-axis of the wearable glass 1000 in the standard wearing state. In this case, it can be seen that an example of a photographed image is shown.

The wearable glasses 1000 according to an embodiment of the present disclosure obtain an image obtained by photographing the user's eyes as information on body parts, and measure the inclination of the wearable glasses 1000 with respect to the user by using the photographed eye image. can decide In this case, the wearable glasses 1000 may directly photograph the user's eyes using a camera provided therein, or an external device receives an image of the user's eyes from an external device to capture an image of the user's eyes. can be obtained

12 is a flowchart of a method for the wearable glasses to determine a tilt of the wearable glasses from an image captured by the user's eyes according to an embodiment of the present disclosure.

In operation S310 , the wearable glasses 1000 according to an embodiment of the present disclosure may determine a first area corresponding to the user's eye from the captured eye image.

For example, the wearable glasses 1000 may determine the first region corresponding to the user's eye based on a change in at least one of brightness and color of the captured eye image.

13 illustrates an example of an image obtained by photographing a user's eyes. The wearable glasses 1000 may determine the area 1301 indicated by the dotted line in FIG. 13 as the first area corresponding to the user's eye.

In operation S320 , the wearable glasses 1000 according to an embodiment of the present disclosure may determine a second region corresponding to the user's pupil from the captured eye image.

For example, the wearable glasses 1000 may determine the second region corresponding to the user's pupil based on a change in at least one of brightness and color of the captured eye image. The wearable glasses 1000 may reduce the amount of computation by determining the second area within the first area determined in step S310 compared to the case of determining the second area within the entire image.

14 illustrates an example of an image obtained by photographing a user's eyes. The wearable glasses 1000 may determine the area 1401 indicated by the dotted line in FIG. 14 as the second area corresponding to the user's pupil.

The wearable glasses 1000 may further determine an area 1403 corresponding to the user's pupil or a center point 1405 of the second area. 14 , the wearable glasses 1000 may further determine a central point 1405 of an area 1403 corresponding to the user's pupil or a center point 1405 of the second area 1401 from the image captured by the user's eyes.

In operation S330 , the wearable glasses 1000 according to an embodiment of the present disclosure may acquire a characteristic value of the user's eyes based on at least one of the first area and the second area. The characteristic value of the user's eye may include at least one of a length value of a major axis, a length value of a minor axis, an angle value in which the major axis is rotated, an angle value in which the minor axis is rotated, and a value indicating the position of the pupil in the image obtained by photographing the eye. may contain one.

15 is a diagram for describing a characteristic value obtained from an eye image captured according to an embodiment of the present disclosure.

15 , the wearable glasses 1000 according to an embodiment of the present disclosure include a straight line passing through two focal points of a first area 1301 corresponding to the user's eye and a first area ( A line segment 1501 connecting points where 1301 intersects may be determined as a major axis, and a length of the line segment 1501 may be determined as a length of a major axis. The two foci of the first region 1301 may mean two foci of the first region approximated by an ellipse when the first region 1301 is approximated to an ellipse.

Also, the wearable glasses 1000 may determine the line segment 1503 that vertically bisects the line segment 1501 as the short axis, and determine the length of the line segment 1503 as the short axis length.

The wearable glasses 1000 may acquire the position of the center 1405 of the second region 1401 corresponding to the user's pupil as a value indicating the position of the pupil.

Meanwhile, the wearable glasses 1000 according to an embodiment of the present disclosure may obtain an angle of a major axis and/or an angle of a minor axis compared with a horizontal axis and/or a vertical axis of an image obtained by photographing an eye. The horizontal axis and the vertical axis of the image obtained by photographing the eye may be the long axis and the short axis of a region corresponding to the eye obtained from the user's eye image photographed in the reference wearing state.

The wearable glasses 1000 may determine an angle 1505 between the long axis of the first region 1301 corresponding to the user's eye and the horizontal axis 1509 - 3 of the image as the long axis angle. Also, the wearable glasses 1000 may determine an angle 1507 formed between the short axis of the first region 1301 corresponding to the user's eye and the vertical axis 1509 - 1 of the image as the angle of the short axis.

In operation S340 , the wearable glasses 1000 according to an embodiment of the present disclosure may determine the inclination of the wearable glasses 1000 with respect to the user based on the acquired characteristic value. For example, the wearable glasses 1000 may determine the inclination by comparing the acquired characteristic value with a reference value included in the reference wearing state information. The reference value included in the reference wearing state information may be a characteristic value detected from an image of a body part obtained when the user wearing the wearable glasses 1000 is the reference wearing state. The reference value may be a predetermined value or a value set by a user.

16 and 17 are diagrams for explaining a method by which wearable glasses determine a tilt of wearable glasses from an eye image according to some embodiments of the present disclosure;

Referring to FIG. 16 , the wearable glasses 1000 according to an embodiment of the present disclosure include a straight line connecting the centers 1405 - 1 and 1405 - 2 of pupils of both eyes of the user and the horizontal axis 1509 - 3) can be extracted as an angle 1601 formed by the characteristic value. The wearable glasses 1000 may determine the vertical inclination of the wearable glasses 1000 by comparing the reference value and the characteristic value.

For example, when the reference value of the angle between the straight line connecting the centers of the pupils of both eyes of the user and the horizontal axis 1509-3 of the image is 0, the centers 1405-1 and 1405 of the pupils of both eyes of the user An angle 1601 between the straight line connecting -2) and the horizontal axis 1509-3 of the image may be determined as the vertical inclination. The wearable glasses 1000 may adjust at least one of a rotation angle, a size, and a shape of the display image based on the determined vertical inclination.

Referring to FIG. 17 , the wearable glasses 1000 according to an embodiment of the present disclosure may extract lengths of long axes 1501-1 and 1501-2 of both eyes of a user as a characteristic value. The wearable glasses 1000 may determine a left and right inclination of the wearable glasses 1000 by comparing the reference value and the characteristic value. For example, the wearable glasses 1000 may store the length of the long axis in advance as a reference value. The wearable glasses 1000 calculate a change in the distance between the wearable glasses 1000 and the user based on the previously stored length of the long axis and the newly measured length of the long axis, and based on the calculated distance, at least the size and shape of the display image. You can adjust one.

18 is a flowchart of a method of displaying, by the wearable glasses 1000 , an image adjusted based on a tilt of the wearable glasses 1000 according to an embodiment of the present disclosure.

In operation S410 , the wearable glasses 1000 according to an embodiment of the present disclosure adjust the display image by rotating the display image based on the tilt of the wearable glasses 1000 . For example, the wearable glasses 1000 may rotate the display image based on a tilt (that is, up and down tilt) rotated based on an axis parallel to the direction of the user's frontal gaze, based on the reference wearing state. have.

The wearable glasses 1000 may determine the inclination of the wearable glasses 1000 in comparison with the reference wearing state, and may rotate the display image by the determined inclination in a direction opposite to the inclination direction of the wearable glasses 1000 .

In operation S450 , the wearable glasses 1000 according to an embodiment of the present disclosure display an image adjusted based on the tilt of the wearable glasses 1000 . The wearable glasses 1000 adjust the image displayed on the display 1030 based on the inclination determined based on the reference wearing state.

20, 21A, and 21B are diagrams for explaining an image displayed through a display based on a tilt of wearable glasses according to some embodiments of the present disclosure; 20 , 21A and 21B , the wearable glasses 1000 according to an embodiment of the present disclosure may display an image through the display 1030 using a prism method. However, the present invention is not limited thereto, and the wearable glasses 1000 according to an embodiment of the present disclosure include the monocular display method illustrated in FIGS. 4A and 4B and the binocular display method illustrated in FIGS. 5A and 5B . and an image may be displayed through the display 1030 using a similar method.

20 illustrates a case in which the user wears the wearable glasses 1000 in the standard wearing state. The wearable glasses 1000 may provide an image 2001 to the user through the display 1030 as shown in FIG. 20 .

21A illustrates a case in which an image displayed through a display is adjusted based on the inclination of the wearable glasses 1000 when the wearable glasses 1000 are tilted compared to the reference wearing state illustrated in FIG. 20 .

When the wearable glasses 1000 are tilted based on an axis parallel to the gaze direction in which the user faces the front as compared to the reference wearing state, the wearable glasses 1000 according to an embodiment of the present disclosure are the wearable glasses 1000 The rotated angle θ can be determined as the vertical inclination. As shown in FIG. 21A , the wearable glasses 1000 may display the image 2101 rotated in the opposite direction by the determined vertical inclination.

19 is a flowchart of a method in which the wearable glasses 1000 display an image adjusted based on a tilt of the wearable glasses 1000 according to an embodiment of the present disclosure.

In operation S420 , the wearable glasses 1000 according to an embodiment of the present disclosure adjust at least one of a size, a rotation angle, and a shape of a display image based on the inclination of the wearable glasses 1000 . For example, the wearable glasses 1000 may adjust the size of a displayed image when the distance from the user to the wearable glasses 1000 is changed based on the reference wearing state. Alternatively, the inclination of the wearable glasses 1000 may be determined based on a predetermined axis compared to the reference wearing state, and the display image may be rotated in a direction opposite to the direction in which the wearable glasses 1000 are inclined. Alternatively, when at least one of a position and an angle of the wearable glasses 1000 is changed based on the reference wearing state, the wearable glasses 1000 may adjust the display image to provide an undistorted image to the user.

In operation S450 , the wearable glasses 1000 according to an embodiment of the present disclosure display an image adjusted based on the tilt of the wearable glasses 1000 .

FIG. 21B shows when the distance from the user to the wearable glasses 1000 increases compared to the reference wearing state shown in FIG. 20 , and when the image displayed through the display is adjusted based on a change in the position of the wearable glasses 1000 shows

When the distance from the user to the wearable glasses 1000 increases compared to the reference wearing state, the user is provided with an image smaller than the image provided in the reference wearing state. Accordingly, the wearable glasses 1000 according to an embodiment of the present disclosure may display an enlarged image 2103 based on the distance at which the wearable glasses 1000 are distant from the user.

21B illustrates a case of wearable glasses using a prism for projecting an image to one eye of a user as an example. However, the wearable glasses 1000 according to an embodiment of the present disclosure are not limited to FIG. 21B . As shown in FIGS. 5A and 5B , in the case of the wearable glasses 1000 configured in the form of a binocular display, if the wearable glasses 1000 are tilted left and right with respect to the user and a left and right inclination exists, one side of the wearable glasses 1000 The distance from the to the user's eye and the distance from the other side of the wearable glasses 1000 to the user's eye are changed.

When the wearable glasses 1000 are tilted from side to side with respect to the user, the wearable glasses 1000 according to an embodiment of the present disclosure may determine the angle at which the wearable glasses 1000 are tilted from side to side with respect to the user as the left and right tilt. .

For example, as shown in FIG. 17 , the wearable glasses 1000 according to an embodiment of the present disclosure are based on the lengths of the long axes 1501-1 and 1501-2 of both eyes of the user. It is possible to determine the left and right slope of (1000). When the wearable glasses 1000 are tilted left and right with respect to the user, the distance from one side of the wearable glasses 1000 to the user's eyes may change. Accordingly, as shown in FIG. 17 , when the distance from one side of the wearable glasses 1000 to the user's predetermined eye increases, the size of the corresponding eye shown in the image obtained by photographing the corresponding eye decreases. In addition, when the distance from the other side of the wearable glasses 1000 to the user's predetermined eye increases, the size of the eye shown in the image obtained by photographing the corresponding eye increases.

The wearable glasses 1000 according to an embodiment of the present disclosure may determine a left/right tilt of the wearable glasses 1000 compared to a reference wearing state, and may adjust the size of a displayed image by the determined left/right tilt. For example, the wearable glasses 1000 may estimate a change in the distance from the wearable glasses 1000 to the user's eyes based on the left and right inclinations. The wearable glasses 1000 may adjust the size of the image being displayed based on the estimated distance change.

In the wearable glasses 1000 according to an embodiment of the present disclosure, necessary image adjustments (eg, horizontal translation, vertical translation), and key stoning are performed based on the tilt of the wearable glasses 1000 . (keystoning, and/or various image processing for correcting so that the user can receive an undistorted image) may be performed.

Hereinafter, a detailed method of adjusting an image displayed through the display by the wearable glasses 1000 according to some embodiments of the present disclosure will be described with reference to FIGS. 22A, 22B and 22C.

The wearable glasses 1000 according to an embodiment of the present disclosure may determine a change in the position of the wearable glasses 1000 with respect to the user in comparison with the reference wearing state. For example, the wearable glasses 1000 may determine the inclination of the wearable glasses 1000 in comparison with the reference wearing state.

The wearable glasses 1000 may adjust an image displayed through the display 1030 based on the determined change in position. The wearable glasses 1000 may display the adjusted image by applying a transformation equation to the image displayed on the screen and re-mapping.

22A , the wearable glasses 1000 according to an embodiment of the present disclosure may display an image 2201 through a display 1030 .

When the user wears the wearable glasses 10 for a long time, the position of the wearable glasses 1000 on the user's head may change according to the user's movement or changes in the external environment. The wearable glasses 1000 may acquire information about a wearing state in which the user is wearing the wearable glasses 1000 . The wearable glasses 1000 may determine the degree of change in the position of the wearable glasses 1000 from wearing state information obtained based on a predetermined reference wearing state. The wearable glasses 1000 may calculate a conversion equation for adjusting the displayed image based on the determined position change.

The conversion equation calculated by the wearable glasses 1000 may be expressed as Equation (1).

In Equation 1, src is the original image before rotation, and dst is the result image after rotation. (x,y) means the coordinates of the image. fx(x,y) denotes a transformation equation for the x-axis value of a predetermined pixel, and fy(x,y) denotes a transformation equation for the y-axis value of a predetermined pixel.

As shown in FIG. 22A , the wearable glasses 1000 apply a transformation equation to the image 2201 displayed on the screen and remap each pixel included in the image 2201 to adjust the image 2203 . can be created and displayed. The wearable glasses 1000 according to an embodiment of the present disclosure may adjust at least one of a position, a rotation angle, a size, and a shape of an image by applying a transformation equation to a displayed image.

22B illustrates a method of adjusting a position of an image provided through the wearable glasses 1000 . 22B , the wearable glasses 1000 according to an embodiment of the present disclosure may display an image 2211 through a display 1030 .

When the user wears the wearable glasses 1000 for a long time, the position of the wearable glasses 1000 on the user's head may change according to the user's movement or changes in the external environment. When the position of the wearable glasses 1000 on the user's head is changed, an image provided through the wearable glasses 1000 may deviate from the user's field of view. In particular, when the display 1030 of the wearable glasses 1000 is configured in a prism shape, when the position of the wearable glasses 1000 on the user's head is changed, the light output from the projector of the display 1030 is focused. As the location is changed, the user is provided with a distorted image.

Accordingly, the wearable glasses 1000 according to an embodiment of the present disclosure determine a change in the position of the wearable glasses 1000 with respect to the user compared to the reference wearing state, and adjust the displayed image based on the determined position change. can The wearable glasses 1000 may calculate a conversion equation for adjusting the displayed image 2211 .

A conversion equation calculated to move the image displayed on the wearable glasses 1000 by k in the x-axis direction and by l in the y-axis direction according to an embodiment of the present disclosure may be expressed as Equation (2).

In Equation 2, src is the original image before changing the position being displayed, and dst is the result image after changing the position. (x,y) means the coordinates of the image. The point 2215 means a predetermined point in the image 2211 being displayed. 22B illustrates a case in which the displayed image 2211 is moved by k in the x-axis direction and by l in the y-axis direction. As shown in FIG. 22B , a predetermined pixel 2215 in an image 2211 that was previously displayed may be moved by k in the x-axis direction and by l in the y-axis direction to be remapped to a pixel 2217 on the screen. have.

As shown in FIG. 22B , the wearable glasses 1000 apply Equation 2 to the image 2211 displayed on the screen and remap each pixel included in the image 2211 to obtain an adjusted image ( 2213 may be generated and displayed. FIG. 22C illustrates a method of rotating an image provided through the wearable glasses 1000 by a predetermined angle. As shown in FIG. 22C , the wearable glasses 1000 according to an embodiment of the present disclosure may display an image 2221 through a display 1030 .

When the user wears the wearable glasses 1000 for a long time, the position of the wearable glasses 1000 on the user's head may change according to the user's movement or changes in the external environment. When the wearable glasses 1000 on the user's head rotate about a predetermined axis, the user is provided with a distorted image.

Accordingly, the wearable glasses 1000 according to an embodiment of the present disclosure may determine the inclination of the wearable glasses 1000 for the user compared to the reference wearing state, and adjust the displayed image based on the determined inclination. . The wearable glasses 1000 may calculate a conversion equation for adjusting the displayed image 2221 . For example, when the wearable glasses 1000 are tilted based on an axis parallel to the gaze direction in which the user faces the front, compared to the reference wearing state, the wearable glasses 1000 according to an embodiment of the present disclosure are the wearable glasses. The angle θ rotated by (1000) may be determined as the vertical inclination. As shown in FIG. 22C , the wearable glasses 1000 may display an image 2221 rotated in the opposite direction by the determined vertical inclination.

A conversion equation calculated to rotate the image displayed on the wearable glasses 1000 in the opposite direction by the vertical inclination according to an embodiment of the present disclosure may be expressed as Equation (3).

A transformation matrix M for rotating the display image in the opposite direction in which the wearable glasses 1000 is tilted may be expressed as follows. That is, M is a transformation matrix for rotating the display image by -θ°.

In Equation 3, center is the center point of the display image to be rotated, and scale is a parameter that determines the size of the resulting image derived by applying a transformation matrix to the display image.

A transformation equation calculated by applying a transformation matrix to a display image may be expressed as Equation (4).

In Equation 4, src is the original image before rotation, and dst is the result image after rotation. (x,y) means the coordinates of the image. As shown in FIG. 22C , the wearable glasses 1000 may generate and display an image 2223 that is rotated by -θ° with respect to the central point 2225 of the displayed image 2221 .

Meanwhile, in the wearable glasses 1000 according to an embodiment of the present disclosure, an animation effect may be applied when an image displayed on the display is updated to an image adjusted based on the inclination of the wearable glasses 1000 and displayed. .

For example, as shown in FIG. 23 , the wearable glasses 1000 display an animation effect indicating that the display image 2301 is rotated and displayed in the direction of the arrow 2305 to be displayed on the image 2301 displayed through the display 1030 . ) can be applied to

After the image adjustment operation based on the inclination of the wearable glasses 1000 is completed, as shown in FIG. 24 , an image 2401 indicating that the image adjustment operation is completed may be output.

Meanwhile, according to an embodiment of the present disclosure, when an image adjusted based on the inclination of the wearable glasses 1000 is out of an area that can be displayed by the wearable glasses 1000, the adjusted image may be reduced and displayed. .

25 is a flowchart of a method for displaying a reduced image of which wearable glasses are adjusted according to an embodiment of the present disclosure;

Steps S200 , S300 , and S400 of FIG. 25 correspond to steps S200 , S300 , and S400 of FIG. 6 , respectively, and thus overlapping descriptions will be omitted.

25 , in step S430 , the wearable glasses 1000 according to an embodiment of the present disclosure adjust an image displayed on the display based on the determined inclination.

In operation S440 , the wearable glasses 1000 according to an embodiment of the present disclosure determine whether the adjusted image is out of a display area of the wearable glasses 1000 .

As an example, in the display-type wearable glasses 1000 shown in FIGS. 4A, 4B, 5A, and 5B, the area that the wearable glasses 1000 can display is light and color corresponding to pixels constituting the image. It may mean an area on the display 1030 that can output . As another example, in the prismatic wearable glasses 1000 shown in FIGS. 3A and 3B , the area that the wearable glasses 1000 can display is a wearable area by reflecting light output from a projector built in the wearable glasses 1000 . It may refer to an area capable of focusing an image on a fovea of a retina of a user's eye wearing the glasses 1000 .

For example, in rotating the display image based on the inclination of the wearable glasses 1000 , when the rotation angle is greater than or equal to a predetermined value, the wearable glasses 1000 may display the adjusted image on the area in which the wearable glasses 1000 can display. can be considered as out of

Alternatively, in changing the size of the display image based on the inclination of the wearable glasses 1000 , when the size of the changed display image is greater than or equal to a predetermined value, the wearable glasses 1000 display the adjusted image to be displayed by the wearable glasses 1000 . It can be judged to be outside the scope of

When the adjusted image is included in the area that can be displayed by the wearable glasses 1000 , the wearable glasses 1000 may display the adjusted image in operation S470 . On the other hand, when the adjusted image is out of a display area of the wearable glasses 1000 , the wearable glasses 1000 may reduce the adjusted image in operation S460 .

26 is a view for explaining an image that is reduced and displayed based on a tilt of wearable glasses according to an embodiment of the present disclosure;

26 is a diagram illustrating a rotation angle of an image displayed on a display based on a vertical inclination of the wearable glasses 1000 with respect to a user when the wearable glasses 1000 are tilted compared to the reference wearing state shown in FIG. 20 . A case of adjustment is shown. At this time, as shown by a dotted line 2601 in FIG. 26 , when the adjusted image is outside the displayable area, the wearable glasses 1000 reduce the adjusted image according to an embodiment of the present disclosure, A reduced image 2603 may be displayed.

On the other hand, when the position of the wearable glasses 1000 with respect to the user is significantly changed compared to the reference wearing state, a distorted image is provided to the user even if the image displayed on the display is adjusted based on the inclination of the wearable glasses 1000 can be In this case, according to an embodiment of the present disclosure, the wearable glasses 1000 may output an image guiding the user to correct the position of the HMD device 1000 .

27 is a flowchart of a method of displaying an image in which wearable glasses notify a user to rewrite wearable glasses according to an embodiment of the present disclosure;

Steps S200 and S300 of FIG. 27 correspond to steps S200 and S300 of FIG. 6 , respectively, and thus overlapping descriptions will be omitted.

27 , in step S300 , the wearable glasses 1000 according to an embodiment of the present disclosure determine the inclination of the wearable glasses 1000 with respect to the user.

In operation S350 , the wearable glasses 1000 according to an embodiment of the present disclosure determine whether the inclination of the wearable glasses 1000 determined in operation S300 is equal to or greater than a predetermined value. In this case, the predetermined value may be a value stored in advance or a value set by the user.

When the determined inclination of the wearable glasses 1000 is less than a predetermined value, in operation S430 , the wearable glasses 1000 may adjust an image displayed on the display based on the determined inclination. On the other hand, when the inclination of the wearable glasses 1000 is equal to or greater than a predetermined value, in step S480 , the wearable glasses 1000 may display an image notifying the user to rewrite the wearable glasses 1000 .

28 illustrates an example of an image displayed to notify a user to rewrite wearable glasses according to an embodiment of the present disclosure.

As shown in FIG. 28 , when the inclination of the wearable glasses 1000 is equal to or greater than a predetermined value, the wearable glasses 1000 display an image 2801 informing the user to rewrite the wearable glasses 1000 on the display 1030 . can be displayed on The image 2801 notifying the user to rewrite the wearable glasses 1000 may include an image providing information on a state in which the wearable glasses 1000 is properly worn to the user (ie, a reference wearing state). .

For example, as shown in FIG. 28 , the image 2801 notifying the user to rewrite the wearable glasses 1000 is such that the state in which the user is wearing the wearable glasses 1000 becomes the reference wearing state. For this purpose, an image 2803 guiding in which direction the wearable glasses 1000 should be rewritten may be included.

A user who is provided with an image notifying that the wearable glasses 1000 needs to be rewritten may physically change the position of the wearable glasses 1000 . The wearable glasses 1000 may display an image notifying the user to rewrite the wearable glasses 1000, then return to step S200 and repeat the operation of adjusting the image based on the information on the user's wearing state. .

Meanwhile, the wearable glasses 1000 according to an embodiment of the present disclosure may display a guide image inducing a user to rewrite the wearable glasses 1000 .

29 illustrates an example of a guide image to induce a user to rewrite wearable glasses, which is displayed according to an embodiment of the present disclosure.

As shown in FIG. 29 , when the inclination of the wearable glasses 1000 is equal to or greater than a predetermined value, the wearable glasses 1000 display an image 2901 informing the user to rewrite the wearable glasses 1000 on the display 1030 . can be displayed on In this case, the image 2901 instructing the user to rewrite the wearable glasses 1000 may include a guide image 2905 inducing the user to rewrite the wearable glasses 1000 .

The guide image 2905 may be an image providing information on the standard wearing state. For example, as shown in FIG. 29 , in the guide image 2905 , the wearable glasses 1000 must be rewritten in any direction so that the state in which the user is wearing the wearable glasses 1000 becomes the reference wearing state. It may be a video that guides you on how to do it.

For example, the guide image 2905 is an image representing a case in which the wearable glasses 1000 are in a horizontal state (that is, a state in which the wearable glasses 1000 are perpendicular to the direction of the earth's gravity) or the wearable glasses 1000 It may be an image expected to be provided to the user when is located at the reference position. The reference position refers to the position of the wearable glasses 1000 relative to the user when the user wears the wearable glasses 1000 as the reference wearing state. It can be determined by value.

For example, in the wearable glasses 1000 according to an embodiment of the present disclosure, based on a built-in sensor, an image estimated to be provided to the user when the wearable glasses 1000 is in a horizontal state is displayed as a guide image 2905 . can be displayed as

Accordingly, as shown in FIG. 29 , the user may physically change the position of the wearable glasses 1000 so that the display image 2903 and the guide image 2905 correspond.

As an example, when the guide image 2905 is an image representing a case in which the wearable glasses 1000 are in a horizontal state, the correspondence between the display image 2903 and the guide image 2905 indicates that the wearable glasses 1000 are in a horizontal state. indicates location. Accordingly, the user may repeat the operation of changing the position of the wearable glasses 100 until the display image 2903 and the guide image 2905 match, so that the wearable glasses 1000 are positioned in a horizontal state. .

As another example, when the guide image 2905 is an image representing a case in which the wearable glasses 1000 are positioned at the reference position, the correspondence between the display image 2903 and the guide image 2905 is that the wearable glasses 1000 are the user indicates that is located in the most suitable position to receive the image. Accordingly, the user may repeat the operation of changing the position of the wearable glasses 100 until the display image 2903 and the guide image 2905 match, so that the wearable glasses 1000 are located at the reference position. .

On the other hand, in the wearable glasses 1000 according to an embodiment of the present disclosure, the position of the wearable glasses 1000 is changed by the user so that the state in which the user is wearing the wearable glasses 1000 becomes the reference wearing state. not limited to The wearable glasses 1000 may include a driving unit (not shown) capable of changing a wearable state of the wearable glasses 1000 by the user. For example, when the wearable glasses 1000 have the same shape as glasses, the wearable glasses 1000 may correct the inclination of the wearable glasses 1000 with respect to the user by including a driving unit (not shown) in the legs or nose pads. have.

On the other hand, the wearable glasses 1000 according to an embodiment of the present disclosure take into consideration not only the inclination of the wearable glasses 1000 but also the position of the wearable glasses 1000 or the change in brightness of the external environment to display an image displayed on the display. Can be adjusted. The wearable glasses 1000 may change the brightness of an image displayed on the display 1030 based on a position of the wearable glasses 1000 or a change in brightness of an external environment. For example, the wearable glasses 1000 may determine a change in brightness of the external environment using an image captured by the user's eyes.

30 is a flowchart of a method of adjusting a display image by wearable glasses according to an embodiment of the present disclosure;

In operation S510 , the wearable glasses 1000 according to an embodiment of the present disclosure may determine at least one of a direction and a distance in which the wearable glasses 1000 move by using an image captured by the user's eyes.

For example, the wearable glasses 1000 may determine the position value of the user's eye from an image obtained by photographing the user's eye. The wearable glasses 1000 may compare the determined position value with a reference value included in the reference wearing state information, and determine at least one of a direction and a distance in which the wearable glasses 1000 moved based on the comparison result.

31 is a view for explaining a method of adjusting a position and size of a display image by wearable glasses according to an embodiment of the present disclosure;

31 shows a first area 1301 corresponding to the user's eye and a second area 1401 corresponding to the user's pupil determined from an image obtained by photographing the user's eye. 31 illustrates an example in which the direction and distance in which the wearable glasses 1000 are moved are determined based on a value indicating the position of the user's pupil. For example, the wearable glasses 1000 according to an embodiment of the present disclosure may determine the position of the center 1405 of the second region 1401 corresponding to the pupil of the user as a value indicating the position of the pupil.

31 , the wearable glasses 1000 compare the pupil position 1405 determined from the image captured by the user's eye with the pre-stored reference position 3101 of the pupil, and based on the comparison result, the wearable glasses (1000) At least one of a moving direction and a distance may be determined. The movement of the wearable glasses 1000 may be expressed as an arrow 3103 connecting the pupil position 1405 determined from the image captured by the user's eye and the pre-stored reference position 3101 of the pupil. In this case, the direction indicated by the arrow 3103 may be the direction in which the wearable glasses 1000 move, and the length of the arrow 3103 may be the distance in which the wearable glasses 1000 are moved.

In operation S520 , the wearable glasses 1000 according to an embodiment of the present disclosure may measure a change in the size of the user's pupil using an image captured by the user's eyes.

For example, the wearable glasses 1000 may determine an area corresponding to the user's pupil from an image captured by the user's eyes, and determine a value indicating the width of the area corresponding to the user's pupil. The wearable glasses 1000 may compare a value determined from an image obtained by photographing the user's eyes with respect to a region corresponding to the user's pupil and a reference value included in the reference wearing state information. The wearable glasses 1000 may measure a change in the size of the user's pupil based on the comparison result.

32 is a view for explaining a method of adjusting the brightness of a display image by wearable glasses according to an embodiment of the present disclosure;

In FIG. 32 , a first area 1301 corresponding to the user's eye determined from an image obtained by photographing the user's eye, a second area 1401 corresponding to the user's pupil, and corresponding to the user's pupil included in the second area A region 1403 is shown.

As shown in FIG. 32 , the wearable glasses 1000 compare an area 1403 corresponding to a pupil determined from an image captured by the user's eye with an area 3201 corresponding to a pre-stored pupil, and based on the comparison result Thus, a change in the size of the user's pupil can be measured. Referring to FIG. 32 , it can be seen that the size of the user's pupil is increased as indicated by an arrow 3203 .

In step S530 , the wearable glasses 1000 according to an embodiment of the present disclosure determine at least one of the location and size of the display image based on at least one of the direction and distance in which the wearable glasses 1000 determined in step S510 . Can be adjusted.

For example, the wearable glasses 1000 may adjust the image displayed on the display to move in a direction opposite to the direction in which the wearable glasses 1000 moved by a distance corresponding to the movement distance of the wearable glasses 1000 . have.

Meanwhile, when the position of the wearable glasses 1000 with respect to the user is changed, the distance from the user's eyes to the wearable glasses 1000 may be changed. Accordingly, the wearable glasses 1000 according to an embodiment of the present disclosure may display a display image in which the distance change is compensated when the distance from the user's eyes to the wearable glasses 1000 is changed compared to the reference wearing state. can For example, when the distance from the user's eyes to the display image increases according to the movement of the wearable glasses 1000, the wearable glasses 1000 may display the display image whose size is changed at a magnification corresponding to the distance. .

In operation S540 , the wearable glasses 1000 according to an embodiment of the present disclosure may adjust the brightness of the display image based on the change in the size of the user's pupil measured in operation S520 .

For example, when the brightness around the wearable glasses 1000 increases, in order for the user to identify the image displayed on the wearable glasses 1000, the brightness of the display image must also increase. In addition, when the brightness around the wearable glasses 1000 is decreased, the brightness of the image displayed on the wearable glasses 1000 should also be decreased in order to prevent glare of the user.

In order to obtain information on the brightness of the external environment of the wearable glasses 1000 , the information may be directly obtained using a brightness sensor provided in the wearable glasses 1000 . Alternatively, information on the brightness of the external environment of the wearable glasses 1000 may be obtained indirectly based on a change in the size of the pupil of the user wearing the wearable glasses 1000 .

The wearable glasses 1000 according to an embodiment of the present disclosure may display a display image whose brightness is adjusted based on a change in the size of the user's pupil. As an example, the wearable glasses 1000, when the size of the user's pupil increases, sees that the brightness of the surroundings has decreased, reduces the brightness of the display image, and when the size of the user's pupil decreases, the brightness of the surroundings decreases. It is reported that it has increased, and the brightness of the display image can be increased.

As another example, the wearable glasses 1000 provide the user with a display image having a brightness that is high enough for the user to identify an image displayed on the wearable glasses 1000 and low enough that the user does not feel glare. should be provided

Therefore, in the wearable glasses 1000 according to an embodiment of the present disclosure, when the size of the user's pupil increases, it is determined that the brightness of the display image is too low and increases the brightness, and when the size of the user's pupil becomes small, It may be determined that the brightness of the display image is too high, and thus the brightness may be decreased.

In FIG. 30 , an example is shown in which the wearable glasses 1000 according to an embodiment of the present disclosure adjust the display image in consideration of both the movement of the wearable glasses 1000 and the size of the pupil, but the present disclosure is limited thereto. doesn't happen The wearable glasses 1000 according to an embodiment of the present disclosure may be configured to adjust at least one of a position, a size, and a brightness of a display image based on at least one of a movement of the wearable glasses 1000 and a size of a pupil. have.

Meanwhile, according to an embodiment of the present disclosure, the wearable glasses illustrated in FIG. 6 may further perform an operation of setting a reference wearing state before wearing the wearable glasses 1000 for a long time. The user may set a state determined to be most suitable for receiving an image from the wearable glasses 1000 as a reference wearing state.

According to an embodiment of the present disclosure, a reference value may be set based on information about a user's body part obtained from the wearable glasses 1000 located at a reference position. The wearable glasses 1000 may store a reference value as reference wearing state information.

For example, when the information on the user's body part includes an image obtained by photographing the user's eyes, the wearable glasses 1000 located at the reference position measure the characteristics of the user's eyes from the image obtained by photographing the user's eyes A value may be set as a reference value.

33 is a view for explaining a method of setting a reference wearing state of wearable glasses according to an embodiment of the present disclosure;

As shown in FIG. 33 , in the wearable glasses 1000 according to an embodiment of the present disclosure, when a user touches at least a part of the wearable glasses 1000 , the wearable glasses 1000 are positioned at a reference position. To do this, it may be determined that the user is physically moving the wearable glasses 1000 .

Accordingly, the wearable glasses 1000 according to an embodiment of the present disclosure may determine the user's wearing state information acquired through the wearable glasses 1000 at the point in time when the user's touch is terminated as the reference wearing state information. The wearable glasses 1000 may update the previously stored reference wearing state information with the newly acquired wearing state information. For example, the wearable glasses 1000 may photograph the user's body part at the point in time when the user's touch is terminated, and store the captured image as wearing state information.

Also, the wearable glasses 1000 according to an embodiment of the present disclosure may prevent the reference wearing state information from being updated when the user unintentionally touches the wearable glasses 1000 . For example, in the wearable glasses 1000 , when a user touches the wearable glasses 1000 , a GUI for receiving confirmation from the user whether to set reference wearing state information based on the current position of the wearable glasses 1000 . (3301) can be output. The wearable glasses 1000 may update the reference wearing state information only when receiving a user input to set the reference wearing state information based on the current position of the wearable glasses 1000 .

Meanwhile, according to an embodiment of the present disclosure, the wearable glasses 1000 may further consider an image reflected on the user's eye when adjusting the image displayed on the display based on the inclination of the wearable glasses 1000 .

A portion of the light output through the display 1030 of the wearable glasses 1000 is absorbed by the user's eye, and the user may recognize the display image from the absorbed light. Also, a portion of light output through the display 1030 of the wearable glasses 1000 may be reflected from the user's eyeball.

As shown in FIG. 34 , according to an embodiment of the present disclosure, the wearable glasses 1000 provide a method of adjusting a display image based on an image reflected from the user's eyeball. According to an embodiment of the present disclosure, the wearable glasses 1000 may include a camera 1060 for photographing the user's eyes. An image obtained by photographing the user's eye by the camera 1060 may include an image 3403 reflected on the user's eye 3401 .

According to an embodiment of the present disclosure, the wearable glasses 1000 may display a test image and adjust the display image based on the reflected image obtained by partially reflecting light constituting the test image from the eyeball.

35 is a flowchart of a method of correcting a display image by wearable glasses based on a test image reflected from a user's eye according to an embodiment of the present disclosure;

In operation S1150, the wearable glasses 1000 according to an embodiment of the present disclosure display a test image. The test image refers to an image used to determine the inclination of the wearable glasses 1000 . For example, the test image may be an image predetermined to be used to determine a slope, or may be an image itself being displayed on a display.

36A, 36B, and 36C show examples of test images used in accordance with some embodiments of the present disclosure.

As shown in FIG. 36A , the wearable glasses 1000 according to an embodiment of the present disclosure may display a test image composed of a plurality of dots.

As shown in FIG. 36B , the wearable glasses 1000 according to an embodiment of the present disclosure may display a test image composed of a plurality of line segments.

As shown in FIG. 36C , the wearable glasses 1000 according to an embodiment of the present disclosure may display a test image including polygons of a predetermined color.

In operation S1200 , the wearable glasses 1000 according to an embodiment of the present disclosure acquires an image obtained by photographing the user's eyes on which the test image is reflected. The wearable glasses 1000 according to an embodiment of the present disclosure may acquire an image obtained by photographing the user's eyes through which the test image is reflected as information on a state in which the user is wearing the wearable glasses 1000 .

For example, the wearable glasses 1000 outputs a test image for a short time so that the user does not recognize it, and acquires an image obtained by photographing the user's eyes on which the test image is reflected, so that the wearable glasses ( 1000) can be controlled.

37A, 37B, and 37C illustrate examples of images obtained by photographing the eyes of a user in which wearable glasses reflect a test image, according to some embodiments of the present disclosure.

As an example, when the wearable glasses 1000 outputs the test image shown in FIG. 36A , the wearable glasses 1000 may photograph the eyes of the user through which the test image is reflected as shown in FIG. 37A .

As another example, when the wearable glasses 1000 outputs the test image shown in FIG. 36B , the wearable glasses 1000 may photograph the eyes of the user through which the test image is reflected as shown in FIG. 37B .

As another example, when the wearable glasses 1000 outputs the test image shown in FIG. 36C , the wearable glasses 1000 may photograph the eyes of the user through which the test image is reflected as shown in FIG. 37C .

The description of S200 of FIG. 6 may be applied to step S1200 of FIG. 35 . Hereinafter, overlapping descriptions will be omitted.

In step S1300 , the wearable glasses 1000 according to an embodiment of the present disclosure determine the inclination of the wearable glasses 1000 with respect to the user by using the captured eye image.

The inclination of the wearable glasses 1000 with respect to the user means the degree of inclination of the wearable glasses 1000 worn by the user based on the reference wearing state. The wearable glasses 1000 may determine the inclination of the wearable glasses 1000 with respect to the user by comparing the image of the user's eyes obtained in step S1200 with the reference wearing state information. The wearable glasses 1000 may store reference wearing state information including an image obtained by photographing the user's eyes obtained in the reference wearing state. The wearable glasses 1000 may store a reflection image of a test image included in an image of an eye obtained in a reference wearing state as reference wearing state information.

The wearable glasses 1000 according to an embodiment of the present disclosure may extract a reflection image of a test image from a photographed eye image. The wearable glasses 1000 may determine the inclination of the wearable glasses 1000 based on at least one of a size and a shape of the reflected image.

For example, the wearable glasses 1000 may determine the inclination of the wearable glasses 1000 with respect to the user by comparing a reflection image extracted from the photographed eye image with a reference image.

Hereinafter, in FIGS. 38, 39a, 39b, 39c, 42, 43a, 43b, 43c, 44a, and 44b, for convenience of explanation, a case in which the rectangular test image shown in FIG. 36B is displayed will be described as an example. However, the present disclosure is not limited thereto, and those skilled in the art can recognize that various test images may be used.

38 is an example of an image 3801 of a user's eyes obtained through the wearable glasses 1000 when the wearable glasses 1000 are not tilted with respect to the user according to an embodiment of the present disclosure.

38 , a photographed eye image 3801 acquired by the wearable glasses 1000 may include a reflection image 3803 of a test image.

For example, the wearable glasses 1000 according to an embodiment of the present disclosure may determine the position of the wearable glasses 1000 photographed in FIG. 38 as a reference position. The wearable glasses 1000 may set and store the eye image 3801 acquired at the reference position as the reference image. Alternatively, the wearable glasses 1000 may set and store the reflection image 3803 acquired at the reference position as the reference image.

Hereinafter, a case in which the reflection image 3805 of FIG. 38 is set as a reference image will be described as an example. FIGS. 39A, 39B, and 39C show when the wearable glasses 1000 are tilted with respect to the user according to some embodiments of the present disclosure. , is an example of an image obtained by photographing the user's eyes obtained through the wearable glasses 1000 .

FIG. 39A illustrates a case in which the wearable glasses 1000 are tilted up and down with respect to the user to measure the vertical inclination. Compared with FIG. 38 , the shape of the reflection image 3803 of FIG. 38 and the shape of the reflection image 3903 of FIG. 39A are the same, but compared with the user's eye 3801 shown in FIG. 38 , the eye ( 3901-1) is inclined. Accordingly, it can be seen that the position of the wearable glasses 1000 with respect to the user has changed.

FIG. 39B illustrates a case in which the wearable glasses 1000 are tilted back and forth with respect to the user so that the front and rear tilt is measured. Compared to the user's eye 3801 shown in FIG. 38 , the position and shape of the eye 3901-2 shown in FIG. 39B are the same, but the shape of the reflection image 3803 of FIG. 38 and the reflection image of FIG. 39B are the same. It can be seen that the shape of (3905) is different. Accordingly, it can be seen that the position of the wearable glasses 1000 with respect to the user has changed.

FIG. 39C illustrates a case in which the wearable glasses 1000 are tilted left and right with respect to the user so that the front and rear tilts are measured. Compared to the user's eye 3801 shown in FIG. 38 , the position and shape of the eyes 3901-3 shown in FIG. 39C are the same, but the shape of the reflection image 3803 of FIG. 38 and the reflection image of FIG. 39C are the same. It can be seen that the shape of (3907) is different. Accordingly, it can be seen that the position of the wearable glasses 1000 with respect to the user has changed.

A method of determining the inclination of the wearable glasses 1000 for the user by using the captured eye image will be described in detail later with reference to FIGS. 40 to 43C .

Returning to FIG. 35 , in step S1400 , the wearable glasses 1000 according to an embodiment of the present disclosure adjust an image displayed on the display based on the determined inclination.

The wearable glasses 1000 may adjust at least one of a rotation angle, a size, and a shape of the display image based on the inclination of the wearable glasses 1000 determined in step S1300 . However, the present disclosure is not limited thereto, and the wearable glasses 1000 according to an embodiment of the present disclosure may adjust various parameters of a display image displayed to a user.

As an example, the wearable glasses 1000 according to an embodiment of the present disclosure may further adjust at least one of a brightness of a display image, a position of an image displayed on the display 1030 , and a color. As another example, when the display 1030 of the wearable glasses 1000 according to an embodiment of the present disclosure is configured with a translucent light wavelength guide (eg, a prism), the wearable glasses 1000 may include the wearable glasses 1000 . A position of a focal point at which light output from the projector of the display 1030 is focused may be adjusted based on the inclination of .

Therefore, a user using the wearable glasses 1000 according to an embodiment of the present disclosure performs a separate operation for correcting the inclination of the wearable glasses 1000 even when the wearable glasses 1000 are tilted with respect to the user. Even if you don't, you can receive undistorted images.

The description of S400 of FIG. 6 may be applied to step S1400 of FIG. 35 . Hereinafter, overlapping descriptions will be omitted.

The wearable glasses 1000 according to an embodiment of the present disclosure display a test image, obtain an image photographed of the user's eyes as wearing state information, extract a reflection image of the test image from the photographed eye image, The inclination of the wearable glasses 1000 with respect to the user may be determined by analyzing the reflection image.

In this case, the wearable glasses 1000 may directly photograph the user's eyes using a camera provided therein, or an external device receives an image of the user's eyes from an external device to capture an image of the user's eyes. can be obtained

40 is a flowchart of a method of determining a tilt of wearable glasses from a photographed eye image according to an embodiment of the present disclosure.

In operation S1310, the wearable glasses 1000 according to an embodiment of the present disclosure may extract a reflection image of the test image from the captured eye image.

For example, the wearable glasses 1000 may determine a first area 1301 corresponding to the user's eye from an image obtained by photographing the user's eye, and may determine the first area 1301 corresponding to the user's pupil included in the first area 1301 . A second region 1401 may be determined. The wearable glasses 1000 may extract the reflection image 4205 of the test image based on at least one of brightness and color in the second region 1301 .

In operation S1320 , the wearable glasses 1000 according to an embodiment of the present disclosure may determine the inclination of the wearable glasses 1000 based on at least one of a size and a shape of the reflected image.

The wearable glasses 1000 according to an embodiment of the present disclosure compare at least one of a size and a shape of a reference image with at least one of a size and a shape of the extracted reflected image, thereby reducing the inclination of the wearable glasses 1000 with respect to the user. can be detected.

41 is a flowchart of a method of displaying an image adjusted based on a tilt of the wearable glasses 1000 by the wearable glasses 1000 according to an embodiment of the present disclosure.

FIG. 12 illustrates a method of acquiring a characteristic value for the eye by analyzing an image captured by the user's eye, and determining the inclination of the wearable glasses 1000 based on the characteristic value of the eye. The method illustrated in FIG. 12 is suitable for determining the vertical inclination of the wearable glasses 1000 . However, in the wearable glasses 1000 performing the method shown in FIG. 41 , by further considering the reflection image of the test image, not only the vertical inclination of the wearable glasses 1000, but also the left and right and front and rear inclinations can be determined with high accuracy.

In operation S1305 , the wearable glasses 1000 according to an embodiment of the present disclosure may determine the vertical inclination of the wearable glasses 1000 .

42 is a diagram for describing characteristic values of a user's eye measured from an eye image captured according to an embodiment of the present disclosure.

As shown in FIG. 42 , the wearable glasses 1000 according to an embodiment of the present disclosure intersect a straight line passing through two focal points of a first area 1301 corresponding to the user's eye and the first area 1301 . A line segment 1501 connecting the points may be determined as a long axis, and a length of the line segment 1501 may be determined as a long axis length.

Also, the wearable glasses 1000 may determine the line segment 1503 that vertically bisects the line segment 1501 as the short axis, and determine the length of the line segment 1503 as the short axis length.

The wearable glasses 1000 may extract the reflection image 4205 of the test image from the second region 1401 corresponding to the user's pupil.

Meanwhile, the wearable glasses 1000 according to an embodiment of the present disclosure may obtain an angle of a major axis and/or an angle of a minor axis compared with a horizontal axis and/or a vertical axis of an image obtained by photographing an eye. When the reflection image is a polygon, the wearable glass 1000 may form an angle between at least one side of the reflection image and a horizontal axis 1509-3 or a vertical axis 1509-1 of the wearable glass 1000 of the wearable glass 1000 . It can be determined as the slope.

The wearable glasses 1000 according to an embodiment of the present disclosure may determine the vertical inclination of the wearable glasses 1000 according to the method illustrated in FIG. 12 . A description overlapping with the description of FIG. 12 will be omitted.

In operation S1310 , the wearable glasses 1000 according to an embodiment of the present disclosure may extract a reflection image of the test image from the photographed eye image.

In operation S1330 , the wearable glasses 1000 according to an embodiment of the present disclosure may determine a left and right inclination and a front and rear inclination of the wearable glasses based on at least one of a size and a shape of the reflected image extracted in operation S1310 .

43A, 43B, and 43C are diagrams for explaining a method of determining a gradient based on a reflection image of a test image included in an eye image in which wearable glasses are photographed, according to some embodiments of the present disclosure;

Referring to FIG. 43A , in the wearable glasses 1000 according to an embodiment of the present disclosure, an angle 4301 between at least one side of a reflection image 4205 and the long axis 1501 of the eye is formed by the wearable glasses 1000 . It can be determined as a vertical inclination.

Referring to FIG. 43B , the wearable glasses 1000 according to an embodiment of the present disclosure may be disposed of before and after the wearable glasses 1000 based on the ratio of the lengths of the two sides 4303 and 4305 included in the reflection image 4205 . slope can be determined. For example, when the test image has a quadrangular shape and the reflection image 4205 has a trapezoidal shape, as the ratio of the lengths of the upper side 4303 and the lower side 4305 increases, the wearable glasses 1000 for the user It can be determined that the anterior-posterior slope of is large.

Referring to FIG. 43C , in the wearable glasses 1000 according to an embodiment of the present disclosure, the left and right sides of the wearable glasses 1000 based on the ratio of the lengths of the two sides 4307 and 4309 included in the reflection image 4205 . slope can be determined. For example, when the test image has a rectangular shape and the reflected image 4205 has a trapezoidal shape rotated by 90 degrees, as the ratio of the length of the upper side 4307 to the lower side 4309 increases, the wearable for the user It may be determined that the left and right inclination of the glass 1000 is large.

In step S1340 , the wearable glasses 1000 according to an embodiment of the present disclosure may display at least one of a rotation angle, a size, and a shape of the display image based on at least one of a vertical tilt, a left/right tilt, and a front/rear tilt of the wearable glass 1000 . You can adjust one. However, the present disclosure is not limited thereto, and the wearable glasses 1000 according to an embodiment of the present disclosure may adjust various parameters of a display image displayed to a user. As an example, the wearable glasses 1000 according to an embodiment of the present disclosure may further adjust at least one of a brightness of a display image, a position displayed on the display 1030, and a color. As another example, when the display 1030 of the wearable glasses 1000 according to an embodiment of the present disclosure is configured with a translucent light wavelength guide (eg, a prism), the wearable glasses 1000 may include the wearable glasses 1000 . A position of a focal point at which light output from the projector of the display 1030 is focused may be adjusted based on the inclination of .

The wearable glasses 1000 according to an embodiment of the present disclosure include necessary image adjustments (eg, horizontal translation) based on at least one of a vertical tilt, a left/right tilt, and a front/rear tilt of the wearable glasses 1000 , vertical translation, keystoning, and/or various image processing for correcting a user to receive an undistorted image) may be performed.

Meanwhile, the wearable glasses 1000 according to an embodiment of the present disclosure may be used to control the operation of the wearable glasses 1000 by analyzing a reflection image of a test image.

44A and 44B are diagrams for explaining a method of operating wearable glasses based on a captured eye image according to some embodiments of the present disclosure;

As shown in FIG. 44A , the wearable glasses 1000 according to an embodiment of the present disclosure include a first area 1301 corresponding to the eye and a second area 1401 corresponding to the pupil from an image captured by the user's eye. ) can be determined. The wearable glasses 1000 may extract a reflection image 4401 of a test image from an image captured by the user's eyes.

As shown in FIG. 44B , in the wearable glasses 1000 according to an embodiment of the present disclosure, when the reflective image 4401 leaves the second area 1401 , the wearable glasses 1000 may perform a predetermined operation. have.

For example, when the user is watching a video through the wearable glasses 1000 , the user may view a place other than the display 1030 of the wearable glasses 1000 . When the user looks at a location other than the display 1030 of the wearable glasses 1000 , the test image may be reflected from a location outside the user's pupil. When the reflection image of the test image is out of the area corresponding to the pupil, the wearable glasses 1000 may stop the reproduction of the video.

The test image may be a predetermined image to be used to control the operation of the wearable glasses 1000 as shown in FIGS. 36A, 36B and 36C , or may be a still image of any one viewpoint included in the moving image.

For example, the wearable glasses 1000 output a test image for a short time so that the user does not recognize it while the video is being played, and acquire an image of the user's eyes in which the test image is reflected, thereby reflecting the test image. The operation of the wearable glasses 1000 may be controlled based on the

Meanwhile, according to an embodiment of the present disclosure, the wearable glasses 1000 may adjust a display image based on a user's gesture. A “gesture” may mean a shape represented by a user's body part at a predetermined time point, a change in a shape represented by the body part for a predetermined time, a change in position, or an operation of the body part.

45 is a diagram for explaining a method of adjusting a display image based on a user's gesture by the wearable glasses 1000 according to an embodiment of the present disclosure.

As shown in FIG. 45 , according to an embodiment of the present disclosure, the wearable glasses 1000 may adjust a display image based on a user's gesture in a space of interest 4501 . For example, the wearable glasses 1000 may adjust the display image based on the gesture of the user's hand 7 .

In order to detect the user's gesture, the wearable glasses 1000 may include a camera or various sensors capable of acquiring an image. Sensors that may be included in the wearable glasses 1000 may include a depth sensor, an infrared sensor, an ultrasonic sensor, and similar sensors. As an example, the wearable glasses 1000 may perform a gesture based on an image of the user's body part or a signal received from the user's body part (eg, an infrared signal or an ultrasonic signal). can be detected. As another example, the wearable glasses 1000 may detect the user's gesture by receiving a signal related to the user's gesture from the wearable device 20 worn on the user's wrist or detecting the movement of the wearable device 20 . have.

46 is a flowchart of a method by which wearable glasses adjust a display image based on a user's gesture according to an embodiment of the present disclosure;

In operation S2100, the wearable glasses 1000 according to an embodiment of the present disclosure may acquire information about a user's gesture.

The wearable glasses 1000 may obtain information about the user's gesture by using a sensing unit provided in the wearable glasses 1000 or may acquire information about the user's gesture from an external device. For example, the wearable glass 1000 includes a magnetic sensor, an acceleration sensor, a gyroscope sensor, a proximity sensor, an optical sensor, a depth sensor, an infrared sensor, and an ultrasonic sensor provided therein. can be used to obtain information about the user's gesture. Alternatively, the wearable glasses 1000 may acquire information about the user's touch gesture received through the user input unit 1040 provided in the wearable glasses 1000 .

The wearable glasses 1000 according to an embodiment of the present disclosure are used to compensate for the inclination of the wearable glasses 1000 when the wearable glasses 1000 are initialized or when the wearable glasses are turned on. information can be obtained. Alternatively, the wearable glasses 1000 may acquire information about the user's gesture when an input of a user who wants to receive an image in which the inclination of the wearable glasses 1000 is compensated is received.

Alternatively, when a large movement of the wearable glasses 1000 occurs, the wearable glasses 1000 may determine that there is a high possibility that the position of the wearable glasses 1000 with respect to the user is changed. Accordingly, when at least one of a motion state value of the wearable glasses 1000, for example, an acceleration value, a velocity value, or each momentum value, is measured to be greater than or equal to a threshold value, the wearable glasses 1000 respond to the user's gesture. information can be obtained.

For example, the information about the user's gesture obtained by the wearable glasses 1000 may acquire at least one of a degree of change in a position of a body part taking a gesture, a degree of change in shape, and a duration of movement.

The information on the user's gesture may be information used to determine an adjustment value for adjusting a display image. For example, the information on the gesture acquired by the wearable glasses 1000 may include information on the user's head, hand, eyes, mouth, or a combination thereof. Also, the information on the gesture acquired by the wearable glasses 1000 may include an image of a body part taking the gesture. Alternatively, the information on the user's gesture may include information on the user's touch input received through the user input unit 1040 .

As an example, the wearable glasses 1000 may acquire information about a gesture based on an image of a hand. The wearable glasses 1000 may acquire an image of a hand, and detect a change in at least one of a palm, a knuckle, and a fingertip of the hand from the acquired image.

As another example, the wearable glasses 1000 may detect a gesture based on a predetermined signal received from a hand. The wearable glasses 1000 may transmit a predetermined signal to the hand, and in response to the transmitted signal, obtain information about the gesture based on a signal reflected from the hand. For example, the predetermined signal may be an infrared signal or an ultrasonic signal. The wearable glasses 1000 may detect a change in at least one of a palm of a hand, a knuckle, and a fingertip from a predetermined signal received from the object.

As another example, the information on the user's gesture may include information on the user input received through the user input unit 1040 . For example, the user input may include an input of touching a touch pad, an input of rotating a wheel, or an input of pushing a button. The wearable glasses 1000 according to an embodiment of the present disclosure may obtain information about the user's touch input received through the user input unit 1040 provided on the wearable glasses 1000 as information about the user's gesture. have. For example, according to some embodiments of the present disclosure, the wearable glasses 1000 may include a touch pad as the user input unit 120 .

A method of acquiring information about a user's gesture will be described in more detail later with reference to FIGS. 47 to 49 .

In operation S2200, the wearable glasses 1000 according to an embodiment of the present disclosure may determine an adjustment value based on the obtained information.

The adjustment value is a value used to adjust the display image. The adjustment value may be a value for adjusting at least one of a rotation angle, a size, and a shape of the display image. Alternatively, the adjustment value may be a value for adjusting the brightness of a display image, a position displayed on the display 1030 , or a color. Alternatively, the adjustment value is the display 1030 of the wearable glasses 1000 when the display 1030 of the wearable glasses 1000 according to an embodiment of the present disclosure is configured as a translucent light wavelength guide (eg, a prism). ) may be a location of a focal point where the light output from the projector is focused.

The wearable glasses 1000 according to an embodiment of the present disclosure may map and store information on various gestures and adjustment values, or store a relational expression capable of calculating an adjustment value based on information on gestures. The wearable glasses 1000 may determine the adjustment value by searching for an adjustment value stored in advance based on the obtained information or calculating the adjustment value using the searched relational expression.

In operation S2300 , the wearable glasses 1000 according to an embodiment of the present disclosure may adjust an image displayed through the display 1030 based on the determined adjustment value.

First, the wearable glasses 1000 according to an embodiment of the present disclosure may adjust at least one of a rotation angle, a size, and a shape of an image displayed through the display 1030 based on the determined adjustment value. Alternatively, the wearable glasses 1000 may adjust at least one of a brightness, a displayed position, and a color of an image displayed through the display 1030 based on the determined adjustment value.

Alternatively, when the display 1030 of the wearable glasses 1000 according to an embodiment of the present disclosure is configured of a translucent light wavelength guide (eg, a prism), the wearable glasses 1000 may Based on the inclination, a position of a focal point at which light output from the projector of the display 1030 is focused may be adjusted.

The wearable glasses 1000 according to an embodiment of the present disclosure provide image adjustments (eg, horizontal translation, vertical translation, keystoning) required based on the adjustment value, and / or various image processing that corrects so that the user can receive an undistorted image) may be performed.

Also, for example, when the display 1030 of the wearable glasses 1000 is configured as a translucent light wavelength guide (eg, a prism), the wearable glasses 1000 display the display based on the inclination of the wearable glasses 1000 . It is possible to adjust the position of the focal point where the light output from the projector of 1030 is focused.

Next, the wearable glasses 1000 according to an embodiment of the present disclosure may display the adjusted image. The description of S400 of FIG. 6 may be applied to step S2300 of FIG. 46 , and overlapping descriptions will be omitted.

When a user's gesture is used to adjust an image displayed through the wearable glasses 1000, an operation of continuously acquiring information about the user's gesture at predetermined time intervals may be performed. For example, in order to obtain information on the user's gesture, the wearable glasses 1000 may photograph the body part of the user making the gesture at predetermined time intervals and analyze the captured image.

However, continuously acquiring information about the user's gesture may result in overload of memory capacity and computational amount of the processor. Accordingly, according to an embodiment of the present disclosure, information on the user's gesture may be acquired only when a predetermined event occurs.

47 is a flowchart of a method by which wearable glasses obtain information about a user's gesture according to an embodiment of the present disclosure;

Information about a user's gesture according to an embodiment of the present disclosure may be information used to determine an adjustment value for adjusting a display image. For example, the information about the gesture may include an image of a body part taking the gesture.

In operation S2110, the wearable glasses 1000 according to an embodiment of the present disclosure determine whether the wearable glasses 1000 are initialized. For example, the wearable glasses 1000 may be initialized when the power is turned on or when an initialization input for initializing the wearable glasses 1000 is received from the user.

The wearable glasses 1000 according to an embodiment of the present disclosure may acquire information about the user's gesture when the wearable glasses 1000 are initialized ( S2150 ). When the wearable glasses 1000 are not initialized (eg, when the wearable glasses 1000 are continuously operating, step S2120 is performed to obtain information on the user's gesture based on the motion state value.

In operation S2120 , the wearable glasses 1000 according to an embodiment of the present disclosure measure a motion state value of the wearable glasses 1000 . The motion state value means a value representing the motion of the wearable glasses 1000 .

For example, the wearable glasses 1000 may measure at least one of an acceleration value, a velocity value, and an angular momentum value as a motion state value. The wearable glasses 1000 may determine whether the movement state value is equal to or greater than a predetermined value (S2130), and when the measured movement state value is greater than or equal to a predetermined value, may obtain information on the user's gesture (S2150). When the motion state value of the wearable glasses 1000 is less than a predetermined value, information on the user's gesture may be obtained based on the user input by performing step S2140.

However, the present disclosure is not limited to step S2130 of FIG. 47 . The wearable glasses 1000 according to an embodiment of the present disclosure may determine whether the motion state value is equal to or greater than a predetermined value, as well as whether the motion state value is equal to or less than a predetermined value or less than a predetermined value. That is, the wearable glasses 1000 according to an embodiment of the present disclosure may compare the motion state value with a predetermined value, and determine whether to acquire information about the user's gesture based on the comparison result.

In operation S2140, the wearable glasses 1000 according to an embodiment of the present disclosure determine whether an input for adjusting a displayed image is received from a user. When the position of the wearable glasses 1000 is changed and a distorted screen is provided, the user may input a command to the wearable glasses 1000 to adjust the display image based on the user's gesture.

The wearable glasses 1000 determine whether a user input for adjusting the display image is received. (S2140) When the user input for adjusting the display image is received, information on the user's gesture may be obtained ( S2150). When the user input of the wearable glasses 1000 is not received, steps S2110 to S2140 may be repeatedly performed.

In operation S2150, the wearable glasses 1000 according to an embodiment of the present disclosure may acquire information about the user's gesture. For example, when the information on the user's gesture is an image obtained by photographing the gesture, the wearable glasses 1000 according to an embodiment of the present disclosure may output a guide image for inducing the user's gesture.

The wearable glasses 1000 may determine an adjustment value for adjusting the display image based on the user's gesture by inducing the user to take a gesture for adjusting the display image by outputting a guide image.

48 and 49 are examples of screens on which the wearable glasses 1000 display a guide image for inducing a user's gesture according to some embodiments of the present disclosure.

48 and 49 , the guide image 4800 displayed through the wearable glasses 1000 may include a virtual input interface 4801 for inducing a user's gesture.

The virtual input interface 4801 may be in the form of a button or a form representing a physical switch (eg, a toggle switch, a rotary switch, a tumbler switch, etc.), and the form may be variously modified.

The user may take a corresponding gesture according to the type of the input interface 4801 provided through the display 1030 of the wearable glasses 1000 .

For example, when receiving the guide image 4800 shown in FIG. 48 , the user may take a gesture of rotating the input interface 4801 with the user's hand. When receiving the guide image 4800 shown in FIG. 49 , the user may take a gesture of pressing at least one arrow-shaped button included in the input interface 4801 with a hand.

In operation S1150, the wearable glasses 1000 according to an embodiment of the present disclosure capture a user's gesture. 52 to 54 show examples of images obtained by photographing a user's gesture obtained through the wearable glasses 1000, according to some embodiments of the present disclosure.

Meanwhile, the wearable glasses 1000 according to an embodiment of the present disclosure may determine an adjustment value for adjusting a display image based on information about a user's gesture. For example, the wearable glasses 1000 may acquire an image of a user's gesture and analyze the captured gesture image to determine an adjustment value.

Hereinafter, a case in which an adjustment value is determined by analyzing a gesture image of a user's hand will be described as an example. However, the present invention is not limited thereto, and the wearable glasses 1000 according to various embodiments of the present disclosure may adjust the display image by using a gesture of a body part other than the hand.

50 is a flowchart of a method of determining an adjustment value from a gesture image captured by wearable glasses according to an embodiment of the present disclosure;

In operation S2210, the wearable glasses 1000 according to an embodiment of the present disclosure may determine an area corresponding to the user's hand from the captured gesture image.

51 is a diagram for explaining a method of detecting a user's hand gesture by the wearable glasses 1000 according to an embodiment of the present disclosure.

As shown in FIG. 51 , the wearable glasses 1000 obtain an image 5100 of a user's hand located within a space of interest located within a predetermined distance from the wearable glasses 1000, and perform a hand gesture from the obtained image. can be detected.

For example, the wearable glasses 1000 may estimate the area 5102 corresponding to the hand from the obtained image 5100 .

In operation S2220, the wearable glasses 1000 according to an embodiment of the present disclosure may detect the number of hands and the number of fingers.

Referring to FIG. 51 , the wearable glasses 1000 analyze the region 5102 corresponding to the estimated hand, and the region 5104 corresponding to the fingertip included in the region 5102 corresponding to the hand and the region corresponding to the back of the hand. Region 5106 may be further detected. However, the present disclosure is not limited to the back of the hand, and a region corresponding to at least a part of the hand (eg, palm, wrist, etc.) may be detected.

The wearable glasses 1000 according to an embodiment of the present disclosure may detect the number of hands and the number of fingers in the ROI based on the detected region.

In operation S2230, the wearable glasses 1000 according to an embodiment of the present disclosure may select a parameter of an image to be adjusted based on the number of hands and the number of fingers. The wearable glasses 1000 according to an embodiment of the present disclosure may determine the type of gesture the user's hand is taking based on the number of hands and the number of fingers.

The wearable glasses 1000 may determine a parameter of an image to be adjusted based on the determined type of gesture. The parameter of the image may include at least one of a size, a shape, and a rotation angle of the image. That is, the wearable glasses 1000 may determine which parameter among the size, position, shape, and rotation angle of the display image to be adjusted based on the determined type of gesture. However, the parameters of the image are not limited to the above-described example.

For example, the parameter of the image may be a parameter related to the brightness of the image, a position where the image is displayed on the display 1030 , or a color of the image. Alternatively, the image parameter is the light output from the projector of the display 1030 of the wearable glass 1000 when the display 1030 of the wearable glass 1000 is configured as a translucent light wavelength guide (eg, a prism). This may be the location of the focused focal point.

In operation S2240, the wearable glasses 1000 according to an embodiment of the present disclosure may detect a change in the position and shape of the hand.

For example, the wearable glasses 1000 may detect a movement (eg, a moving distance or an angle) of the area 5104 corresponding to the fingertip with respect to the area 5106 corresponding to the back of the hand.

For example, the wearable glasses 1000 may detect a motion in which the user bends a finger by detecting that the area 5106 corresponding to the back of the hand and the area 5104 corresponding to the tip of the finger come closer. Alternatively, the wearable glasses 1000 may detect a motion in which the user spreads a finger by detecting that the area 5106 corresponding to the back of the hand and the area 5104 corresponding to the tip of the finger move away from each other.

In operation S2250 , the wearable glasses 1000 according to an embodiment of the present disclosure may determine an adjustment value for adjusting the selected parameter based on a change in the position and shape of the hand.

For example, the wearable glasses 1000 may determine the adjustment value for how much to adjust the selected parameter based on the distance that at least a part of the hand has moved for a predetermined time or the degree to which the shape is changed. However, the present disclosure is not limited thereto. For example, the wearable glasses 1000 may provide an adjustment value based on various information obtained based on a gesture, such as the speed at which the hand moves, the position at which the hand is detected, the direction in which the hand moves, or the number of repetitions of an action taking a predetermined gesture. can be decided

52, 53, and 54 are diagrams for explaining a method of determining, by wearable glasses, an adjustment value for adjusting a display image based on a user's gesture, according to some embodiments of the present disclosure;

Referring to FIG. 52 , the wearable glasses 1000 may output a guide image guiding the user to take a gesture of rotating the virtual input interface 4801 with a hand 7 .

When a gesture of the user's hand 7 corresponding to the displayed virtual input interface 4801 is detected, the wearable glasses 1000 may change the movement angle, movement distance, or movement speed of the user's hand 7 . Based on this, the display image may be adjusted.

For example, when the user makes a gesture of rotating the virtual input interface 4801 clockwise with the hand 7 , the wearable glasses 1000 are displayed clockwise by the rotation angle of the user's hand 7 . You can rotate the image.

53 , the wearable glasses 1000 may output a guide image that induces the user to press at least one button included in the virtual input interface 4801 with the hand 7 . .

When a gesture of the user's hand 7 corresponding to the displayed virtual input interface 4801 is detected, the wearable glasses 1000 determine the position of the user's hand 7 and the distance the user's hand 7 moves. Alternatively, the display image may be adjusted based on the number of movements.

For example, when the user makes a gesture of pressing a button in the form of an arrow included in the virtual input interface 4801 with his hand 7 , the wearable glasses 1000 move the display image in a direction corresponding to the button in the form can do it In this case, the wearable glasses 1000 may move the display image by a distance corresponding to the number of times the user's hand 7 has moved.

As shown in FIG. 54 , the wearable glasses 1000 may output a guide image inducing the user to make a gesture of moving the virtual input interface 4801 with a hand 7 .

When a gesture of the user's hand 7 corresponding to the displayed virtual input interface 4801 is detected, the wearable glasses 1000 may determine the distance the user's hand 7 moves or the angle at which the hand 7 rotates. The display image may be adjusted based on the

For example, when the user makes a gesture of rotating the virtual input interface 4801 with his hand 7 , the wearable glasses 1000 may rotate the display image by the rotation angle of the user's hand 7 . .

A method of adjusting a display image based on a gesture in a user's space of interest in accordance with some embodiments of the present disclosure has been described above with reference to FIGS. 50 to 54 . However, the present disclosure is not limited thereto.

For example, according to some embodiments of the present disclosure, the wearable glasses 1000 may display information about the user's touch input received through the user input unit 1040 provided on the wearable glasses 1000 for the user's gesture. information can be obtained. For example, according to some embodiments of the present disclosure, the wearable glasses 1000 may include a touch pad as the user input unit 120 . The touch pad provided in the wearable glass 1000 may include a contact-type capacitive method, a pressure-type resistive film method, an infrared sensing method, a surface ultrasonic conduction method, an integral tension measurement method, a piezo effect method, and the like, but is limited thereto. it is not

According to some embodiments of the present disclosure, the touch pad provided in the wearable glasses 1000 may be configured to detect a proximity touch as well as a user's real-touch with respect to the touch pad. . As used herein, the term “real-touch” refers to a case in which a pointer is actually touched on the screen, and “proximity-touch” refers to a case in which the pointer actually touches the screen. It refers to a case in which the screen is approached at a predetermined distance away from the screen.

In the present specification, a pointer refers to a tool for touching or close-touching a specific part of a displayed screen. Examples include a stylus fan, a finger, and the like.

Although not shown in the drawings, various sensors may be provided inside or near the touch pad to sense a touch or a proximity touch of the touch pad. A tactile sensor is an example of a sensor for sensing a touch of a touch pad. A tactile sensor refers to a sensor that senses a touch of a specific object to the extent or higher than that of a human being. The tactile sensor may sense various information such as the roughness of the contact surface, the hardness of the contact object, and the temperature of the contact point.

Also, as an example of a sensor for detecting a touch of a touch pad, there is a proximity sensor. The proximity sensor refers to a sensor that detects the presence or absence of an object approaching a predetermined detection surface or an object existing in the vicinity without mechanical contact using the force of an electromagnetic field or infrared rays. Examples of the proximity sensor include a transmission type photoelectric sensor, a direct reflection type photoelectric sensor, a mirror reflection type photoelectric sensor, a high frequency oscillation type proximity sensor, a capacitive type proximity sensor, a magnetic type proximity sensor, and an infrared proximity sensor.

Meanwhile, the user's touch gesture may include a tap, a touch & hold, a double tap, a drag, a pan, a flick, a drag and drop, a swipe, and the like.

The wearable glasses 1000 according to an embodiment of the present disclosure may acquire information about a user's gesture through the user input unit 1040 . For example, the wearable glasses 1000 may sense a touch input for adjusting a display image through the user input unit 1040 . For example, the user may drag or tap a predetermined point on the touch pad of the user input unit 1040 .

“Drag” refers to an operation in which a user touches a finger or a touch tool on the screen and then moves the finger or the touch tool to another location on the screen while maintaining the touch. A “tap” refers to an action of a user touching a screen using a finger or a touch tool (eg, an electronic pen) and then immediately lifting it off the screen without moving.

The wearable glasses 1000 may adjust a display image based on a user's touch input. For example, the wearable glasses 1000 may obtain a user's touch input as information about a user's gesture, determine an adjustment value based on the user's touch input, and adjust a display image based on the determined adjustment value. have.

55 and 56 are diagrams for explaining a method by which wearable glasses adjust a display image based on a user's gesture according to some embodiments of the present disclosure;

55 and 56 illustrate a case in which the wearable glasses 1000 include a touch pad as the user input unit 1040 according to some embodiments of the present disclosure. The touch pad provided in the wearable glass 1000 may include a contact-type capacitive method, a pressure-type resistive film method, an infrared sensing method, a surface ultrasonic conduction method, an integral tension measurement method, a piezo effect method, and the like, but is limited thereto. it is not

First, as shown in FIG. 55 , the wearable glasses 1000 may detect a gesture in which the user drags a predetermined point on the touch pad of the user input unit 1040 . The wearable glasses 1000 may adjust the display image based on at least one of the direction, duration, and distance of the user's drag gesture.

For example, as shown in FIG. 55 , the wearable glasses 1000 may change the position of the display image 5501 based on a user's drag input.

Meanwhile, the wearable glasses 1000 according to an embodiment of the present disclosure may identify a multi-touch by recognizing a plurality of touch points through the user input unit 1040 . The wearable glasses 1000 according to an embodiment of the present disclosure may set the wearable glasses 1000 to react differently according to the number of sensed touch points by identifying multi-touch.

56 , the wearable glasses 1000 may detect a gesture in which the user touches a predetermined point on the touch pad of the user input unit 1040 using a plurality of fingers. The wearable glasses 1000 may adjust the display image based on at least one of the number of the user's touch points, the moving direction, and the duration and distance of the touch.

For example, as shown in FIG. 56 , the wearable glasses 1000 may display a display image ( 5501) can be rotated.

However, the present disclosure is not limited to FIGS. 55 and 56 . The wearable glasses 1000 according to an embodiment of the present disclosure may adjust at least one of a rotation angle, a size, a position, a brightness, and a shape of a display image according to a user's various touch inputs. Also, for example, when the display 1030 of the wearable glasses 1000 is configured as a translucent light wavelength guide (eg, a prism), the wearable glasses 1000 display the display 1030 based on the user's touch input. You can adjust the position of the focal point where the light output from the projector is focused.

Meanwhile, the wearable glasses 1000 according to an embodiment of the present disclosure may operate in connection with a device such as a mobile phone, a PDA, or a laptop. The device 2000 may display an image through the wearable glasses 1000 .

57 is a diagram illustrating a system in which a device displays an image through wearable glasses according to an embodiment of the present disclosure;

Referring to FIG. 57 , the wearable glasses 1000 may provide information used to adjust a display image to the device 2000 .

The information provided by the wearable glasses 1000 to the device 2000 may include, for example, information on a body part of a user wearing the wearable glasses 1000 , information on a user's gesture, or the wearable glasses 1000 . It may include a user input input through the

The device 2000 adjusts a display image based on information received from the wearable glasses 1000 , and provides the adjusted image to the wearable glasses 1000 . The device 2000 may display the adjusted image through the wearable glasses 1000 .

The wearable glasses 1000 may photograph a body part of the user and transmit the photographed image to the device 2000 . For example, the wearable glasses 1000 may photograph the user's eyes or hands.

Also, the device 2000 may receive an image photographed through the wearable glasses 2000 from the wearable glasses 1000 , and may determine a display image through the wearable glasses 1000 . Also, the device 2000 may determine a position at which an image is to be displayed, and display the image at a predetermined position through the wearable glasses 1000 .

In addition, the device 2000 includes a smart phone, a tablet PC, a PC, a smart TV, a mobile phone, a personal digital assistant (PDA), a laptop, a media player, a micro server, a global positioning system (GPS) device, an e-book terminal, and a digital broadcasting terminal. , navigation, kiosks, MP3 players, digital cameras, and other mobile or non-mobile computing devices. Also, the device 2000 may include various devices capable of receiving a touch input, such as an electronic blackboard, a touch table, and the like. Also, the device 2000 may be a wearable device having a communication function and a data processing function. However, the present invention is not limited thereto, and the device 2000 may include any type of device capable of transmitting and receiving data from the wearable glasses 1000 through a network.

The device 2000 and the wearable glasses 1000 may be connected through a network, and the network may include a local area network (LAN), a wide area network (WAN), or a value added network (VAN). It may be implemented in all types of wireless networks, such as a wired network, such as a mobile radio communication network, or a satellite communication network.

58 is a flowchart of a method for a device to display an image through wearable glasses according to an embodiment of the present disclosure;

In operation S3100 , the device 2000 according to an embodiment of the present disclosure may receive information about a state in which the user wears the wearable glasses 1000 from the wearable glasses 1000 . The information on the state in which the user is wearing the wearable glasses 1000 may include information on the user's body part. The information on the user's body part may be information used to determine the inclination of the wearable glasses 1000 with respect to the user in comparison with the reference wearing state. For example, the information on the body part obtained by the wearable glasses 1000 may include information on the user's eyes, nose, ears, mouth, hands, or a combination thereof. Also, the information on the body part obtained by the wearable glasses 1000 may include an image of the body part.

In operation S3100 , the information on the wearing state received by the device 2000 from the wearable glasses 1000 may be information obtained from the wearable glasses 1000 . Since step S200 of FIG. 58 and step S200 of FIG. 6 correspond to each other, overlapping descriptions will be omitted.

In operation S3200, the device 2000 according to an embodiment of the present disclosure may determine the inclination of the wearable glasses 1000 for the user by using the received information on the body part.

The inclination of the wearable glasses 1000 with respect to the user means the degree of inclination of the wearable glasses 1000 determined based on the position of the wearable glasses 1000 most suitable for the user to receive an image.

For example, the device 2000 may determine the inclination of the wearable glasses 1000 with respect to the user by comparing the predetermined reference wearing state information with information received from the wearable glasses 1000 . The reference wearing state information is information indicating a location of the wearable glasses 1000 most suitable for a user to receive an image, and may be pre-stored or set by the user.

For example, the wearing state information may include an image of a body part of the user. The device 2000 may detect a region corresponding to the body part from the image of the body part, and obtain a characteristic value of the body part from the detected region. The device 2000 may acquire a characteristic value related to a location, shape, or size of a region corresponding to a body part detected in an image of the body part. The device 2000 may determine the inclination by comparing the acquired characteristic value with a reference value included in the reference wearing state information. The reference value included in the reference wearing state information may be a characteristic value detected from an image of a body part obtained in the reference wearing state.

In operation S3300 , the device 2000 according to an embodiment of the present disclosure may adjust an image displayed through the display 1030 of the wearable glasses 1000 based on the inclination determined in operation S3200 . For example, the device 2000 may adjust at least one of a rotation angle, a size, and a shape of the display image based on the inclination determined in step S3200 .

However, the present disclosure is not limited thereto, and the device 2000 according to an exemplary embodiment may adjust various parameters of an image provided to a user through the display 1030 . As an example, the device 2000 according to an embodiment of the present disclosure may further adjust at least one of a brightness, a displayed position, and a color of an image displayed through the display 1030 .

As another example, when the display 1030 of the wearable glasses 1000 according to an embodiment of the present disclosure is configured as a translucent light wavelength guide (eg, a prism), the device 2000 is the wearable glasses 1000 . Based on the inclination, it is possible to adjust the position of the focus where the light output from the projector of the display 1030 is focused.

In operation S3400 , the device 2000 according to an embodiment of the present disclosure may provide the image adjusted in operation S3300 to the wearable glasses 1000 . The adjusted image provided from the device 2000 to the wearable glasses 1000 may be displayed through the wearable glasses 1000 ( S400 ).

When the device 2000 according to an embodiment of the present disclosure adjusts a display image using information on body parts, the method described above with reference to FIGS. 6 to 44B may be applied. That is, in the method for the wearable glasses 1000 to adjust a display image according to an embodiment of the present disclosure described above with reference to FIGS. 6 to 44B , according to an embodiment of the present disclosure, the device 2000 uses the wearable glasses ( 1000) may also be applied to the case of adjusting the displayed image. Accordingly, overlapping descriptions are omitted.

59 is a flowchart of a method for a device to display an image through wearable glasses according to an embodiment of the present disclosure;

In operation S4100 , the device 2000 according to an embodiment of the present disclosure may receive information about the user's gesture from the wearable glasses 1000 .

The information on the user's gesture may be information used to determine an adjustment value for adjusting a display image. For example, the information on the gesture acquired by the wearable glasses 1000 may include information on the user's head, hand, eyes, mouth, or a combination thereof. Also, the information on the gesture acquired by the wearable glasses 1000 may include an image of a body part taking the gesture.

The information on the gesture that the device 2000 receives from the wearable glasses 1000 in operation S4100 may be information obtained from the wearable glasses 1000 . Since step S2100 of FIG. 59 and step S2100 of FIG. 46 correspond to each other, overlapping descriptions will be omitted.

In operation S4200, the device 2000 according to an embodiment of the present disclosure may determine an adjustment value for adjusting the display image by using the received gesture information.

For example, the adjustment value may be a value for adjusting at least one of a rotation angle, a size, and a shape of a display image. Alternatively, the adjustment value may be a value for adjusting the brightness of a display image, a position displayed on the display 1030 , or a color. Alternatively, the adjustment value is the display 1030 of the wearable glasses 1000 when the display 1030 of the wearable glasses 1000 according to an embodiment of the present disclosure is configured as a translucent light wavelength guide (eg, a prism). ) may be a location of a focal point where the light output from the projector is focused.

The device 2000 may map and store information on various gestures and adjustment values, or store a relational expression capable of calculating an adjustment value based on information on the gesture. The device 2000 may determine the adjustment value by searching for an adjustment value stored in advance based on the obtained information or calculating the adjustment value using the searched relational expression.

In operation S4300 , the device 2000 according to an embodiment of the present disclosure may adjust the display image based on the determined adjustment value. For example, the wearable glasses 1000 may adjust at least one of a rotation angle, a size, and a shape of the display image based on the determined adjustment value.

In operation S4400 , the device 2000 according to an embodiment of the present disclosure may provide the image adjusted in operation S4300 to the wearable glasses 1000 . The adjusted image provided from the device 2000 to the wearable glasses 1000 may be displayed through the wearable glasses 1000 ( S400 ).

When the device 2000 according to an embodiment of the present disclosure adjusts a display image using information about a user's gesture, the method described above with reference to FIGS. 45 to 56 may be applied. That is, in the method for the wearable glasses 1000 to adjust a display image according to an embodiment of the present disclosure described above with reference to FIGS. 45 to 56 , according to an embodiment of the present disclosure, the device 2000 uses the wearable glasses ( 1000) may also be applied to the case of adjusting the displayed image. Accordingly, overlapping descriptions are omitted.

60 is a block diagram of wearable glasses according to an embodiment of the present disclosure.

As shown in FIG. 60 , the wearable glasses 1000 according to an embodiment may include a sensing unit 1100 , a processor 1200 , and a display 1030 . However, the wearable glasses 1000 may be implemented by more components than those shown in FIG. 60 . In addition, each component included in the wearable glasses 1000 may be disposed on a frame used to wear the wearable glasses 1000 on a user's head.

The display 1030 displays an image to a user. The display 1030 displays information processed by the wearable glasses 1000 . The display 1030 may display an image adjusted based on the inclination of the wearable glasses 1000 or the user's gesture.

In addition, the display 1030 provides a user interface for setting a reference value used to adjust the display image, a user interface for inducing a user's gesture used to adjust the display image, or information related to adjustment of the display image to the user. A user interface can be displayed.

The sensing unit 1100 may acquire information about a user's body part or information about a user's gesture. The information on the user's body part may include an image of the body part, and the information on the user's gesture may include an image of the user's body part taking the gesture.

For example, the sensing unit 1100 may acquire an image obtained by photographing the user's eyes or an image obtained by photographing the user's hand.

The processor 1200 according to an embodiment of the present disclosure may determine the inclination of the wearable glasses for the user by using information on the user's body part, and adjust the display image based on the determined inclination. Alternatively, the processor 1200 according to an embodiment of the present disclosure may determine an adjustment value using information about the user's gesture, and adjust the display image based on the adjustment value.

In addition, as shown in FIG. 61 , the wearable glasses 1000 according to an embodiment of the present disclosure include a communication unit 1300 , a memory 1400 , a user input unit 1040 , an output unit 1500 , and a power supply unit ( 1600) may be further included. According to an embodiment of the present disclosure, the sensing unit 1100 may include at least one camera 1050 , 1060 , 1070 and a sensor 1150 . The various components described above may be connected to each other through a bus.

Hereinafter, the above components will be described in turn.

The cameras 1050 , 1060 , and 1070 capture objects in real space. The object images captured by the cameras 1050 , 1060 , and 1070 may be moving images or continuous still image images. The wearable glasses 1000 may be, for example, a device in the form of glasses having a communication function and a data processing function. You can photograph objects in space.

Also, the camera 1060 may photograph the user's eyes. For example, the camera 1060 facing the user's face in the wearable glasses 1000 worn by the user may photograph the user's eyes.

Also, the eye tracking camera 1070 may photograph the user's eyes. For example, the eye tracking camera 1070 facing the user's face in the wearable glasses 1000 worn by the user tracks at least one of the user's head pose, eyelids, and pupils of the user. You can follow your gaze.

The sensor 1150 may detect a state of the wearable glasses 1000 or a state around the wearable glasses 1000 , and transmit the sensed information to the processor 1200 . For example, the sensor 1150 may acquire information about a wearing state in which the user is wearing the wearable glasses 1000 . For example, the sensor 1150 may include a geomagnetic sensor, an acceleration sensor, a gyroscope sensor, a proximity sensor, an optical sensor, a depth sensor, an infrared sensor, and an ultrasonic sensor.

The communication unit 1300 may transmit/receive information necessary for the wearable glasses 1000 to display an image and to adjust the display image with the device 2000, a peripheral device, or a server.

The memory 1400 stores information necessary for the wearable glasses 1000 to display an image and to adjust an image based on the tilt of the wearable glasses 1000 . The memory 1400 may store information on a reference wearing state, which is a state in which the wearable glasses 1000 are worn at a position most suitable for a user to receive an image. For example, the memory 1400 may store reference wearing state information including a characteristic value detected from an image of a user's body part obtained in a standard wearing state and an image of a body part acquired in a reference wearing state. have.

The user input unit 1040 receives a user input for controlling the wearable glasses 1000 . The user input unit 1040 may receive a touch input and a key input for the wearable glasses 1000 . Also, the user input unit 1040 may receive a user's gesture input photographed from the camera 1050 .

The power supply unit 1600 supplies power required for the operation of the wearable glasses 1000 to each component. The power supply 1600 may include a battery (not shown) capable of charging power, and may include a cable (not shown) or a cable port (not shown) capable of receiving power from the outside.

The output unit 1500 outputs information received from the communication unit 1300 , processed by the processor 1200 , or stored in the memory 1400 in the form of at least one of light, sound, and vibration. For example, the output unit 1500 may include a speaker 1020 that outputs audio data. Also, the speaker 1020 may output a sound signal related to a function (eg, a call signal reception sound, a message reception sound, and a notification sound) performed by the wearable glasses 1000 .

The processor 1200 may control the overall operation of the wearable glasses 1000 . For example, the processor 1200 executes programs stored in the memory 1400 , so that the display 1030 , the sensing unit 1100 , the communication unit 1300 , the memory 1400 , the user input unit 1040 , and the output unit 1500 , and the power supply 1600 may be controlled.

The wearable glasses 1000 may be connected to the device 2000 , and may display the image on the display 1030 of the wearable glasses 1000 by receiving information about the display image from the device 2000 .

62 and 63 are block diagrams of a device 2000 in accordance with some embodiments of the present disclosure.

62 , the device 2000 according to some embodiments may include a user input unit 6100 , an output unit 6200 , a control unit 6300 , and a communication unit 6500 . However, not all of the components shown in FIG. 62 are essential components of the device 2000 . The device 2000 may be implemented by more components than the components shown in FIG. 62 , or the device 2000 may be implemented by fewer components than the components shown in FIG. 62 .

For example, as shown in FIG. 63 , the device 2000 according to some embodiments includes a sensing unit 6400 in addition to the user input unit 6100 , the output unit 6200 , the control unit 6300 , and the communication unit 6500 . ), an A/V input unit 6600 , and a memory 6700 may be further included.

The user input unit 6100 means a means for a user to input data for controlling the device 2000 . For example, the user input unit 6100 includes a key pad, a dome switch, and a touch pad (contact capacitive method, pressure resistance film method, infrared sensing method, surface ultrasonic conduction method, integral type). There may be a tension measurement method, a piezo effect method, etc.), a jog wheel, a jog switch, and the like, but is not limited thereto.

The user input unit 6100 may receive a user input for controlling the wearable glasses 1000 . Also, the user input unit 6100 may receive a user input for adjusting an image displayed through the wearable glasses 1000 based on information about a user's body part or information about a gesture.

The output unit 6200 may output an audio signal, a video signal, or a vibration signal, and the output unit 6200 may include a display unit 6210 , a sound output unit 6220 , and a vibration motor 6230 . have.

The display unit 6210 displays and outputs information processed by the device 2000 . For example, the display unit 6210 receives a user interface for receiving a user input for controlling the wearable glasses 1000 , and a setting value related to an operation of adjusting an image displayed through the wearable glasses 1000 . can display a user interface for

On the other hand, when the display unit 6210 and the touch pad form a layer structure to form a touch screen, the display unit 6210 may be used as an input device in addition to an output device. The display unit 6210 includes a liquid crystal display, a thin film transistor-liquid crystal display, an organic light-emitting diode, a flexible display, a three-dimensional display ( 3D display) and electrophoretic display (electrophoretic display) may include at least one. Also, depending on the implementation of the device 2000 , the device 2000 may include two or more display units 6210 . In this case, two or more display units 6210 may be disposed to face each other using a hinge.

The sound output unit 6220 outputs audio data received from the communication unit 6500 or stored in the memory 6700 . Also, the sound output unit 6220 outputs a sound signal related to a function (eg, a call signal reception sound, a message reception sound, and a notification sound) performed by the device 2000 . The sound output unit 6220 may include a speaker, a buzzer, and the like.

The vibration motor 6230 may output a vibration signal. For example, the vibration motor 6230 may output a vibration signal corresponding to the output of audio data or video data (eg, a call signal reception sound, a message reception sound, etc.). Also, the vibration motor 6230 may output a vibration signal when a touch is input to the touch screen.

The controller 6300 generally controls the overall operation of the device 2000 . For example, the control unit 6300 executes programs stored in the memory 6700 , and thus the user input unit 6100 , the output unit 6200 , the sensing unit 6400 , the communication unit 6500 , and the A/V input unit 6600 . ) can be controlled in general.

Specifically, the controller 6300 receives information used to adjust a display image from the wearable glasses 1000 by controlling the communication unit 6500 , and adjusts the display image based on the received information.

Also, the controller 6300 receives a user input for controlling the operation of the wearable glasses 1000 . The user input may receive, for example, a touch input, a button input, and a voice input, but is not limited thereto. Also, the controller 6300 may receive a user input for the wearable glasses 1000 from the wearable glasses 1000 .

The sensing unit 6400 may detect a state of the device 2000 or a state around the device 2000 , and transmit the sensed information to the controller 6300 .

The sensing unit 6400 includes a magnetic sensor 6410 , an acceleration sensor 6420 , a temperature/humidity sensor 6430 , an infrared sensor 6440 , a gyroscope sensor 6450 , and a position sensor. (eg, GPS) may include at least one of a 6460 , a barometric pressure sensor 6470 , a proximity sensor 6480 , and an illuminance sensor 6490 , but is not limited thereto. Since a function of each sensor can be intuitively inferred from the name of a person skilled in the art, a detailed description thereof will be omitted.

The communication unit 6500 may include one or more components that enable communication between the device 2000 and the wearable glasses 1000 or between the device 2000 and a server (not shown). For example, the communication unit 6500 may include a short-range communication unit 6510 , a mobile communication unit 6520 , and a broadcast receiving unit 6530 .

Short-range wireless communication unit 65110, Bluetooth communication unit, BLE (Bluetooth Low Energy) communication unit, short-range wireless communication unit (Near Field Communication unit), WLAN (Wi-Fi) communication unit, Zigbee (Zigbee) communication unit, infrared ( It may include an IrDA, infrared Data Association) communication unit, a Wi-Fi Direct (WFD) communication unit, an ultra wideband (UWB) communication unit, and the like, but is not limited thereto.

The mobile communication unit 6520 transmits/receives wireless signals to and from at least one of a base station, an external terminal, and a server on a mobile communication network. Here, the wireless signal may include various types of data according to transmission and reception of a voice call signal, a video call signal, or a text/multimedia message.

The broadcast receiver 6530 receives a broadcast signal and/or broadcast-related information from the outside through a broadcast channel. The broadcast channel may include a satellite channel and a terrestrial channel. According to an implementation example, the device 2000 may not include the broadcast receiver 6530 .

The A/V (Audio/Video) input unit 600 is for inputting an audio signal or a video signal, and may include a camera 6610 and a microphone 61620 . The camera 6610 may obtain an image frame such as a still image or a moving image through an image sensor in a video call mode or a shooting mode. The image captured through the image sensor may be processed through the controller 6300 or a separate image processing unit (not shown).

The image frame processed by the camera 6610 may be stored in the memory 6700 or transmitted to the outside through the communication unit 6500 . Two or more cameras 6610 may be provided according to the configuration of the terminal.

The microphone 6620 receives an external sound signal and processes it as electrical voice data. For example, the microphone 6620 may receive an acoustic signal from an external device or a speaker. The microphone 6620 may use various noise removal algorithms for removing noise generated in the process of receiving an external sound signal.

The memory 6700 may store a program for processing and control of the controller 6300 , and may store data input to or output from the device 2000 .

The memory 6700 may include a flash memory type, a hard disk type, a multimedia card micro type, a card type memory (eg, SD or XD memory), and a RAM. (RAM, Random Access Memory) SRAM (Static Random Access Memory), ROM (Read-Only Memory), EEPROM (Electrically Erasable Programmable Read-Only Memory), PROM (Programmable Read-Only Memory), magnetic memory, magnetic disk , may include at least one type of storage medium among optical disks.

Programs stored in the memory 6700 may be classified into a plurality of modules according to their functions, for example, may be classified into a UI module 6710 , a touch screen module 6720 , a notification module 6730 , and the like. .

The UI module 6710 may provide a specialized UI, GUI, or the like that is interlocked with the device 2000 for each application. The touch screen module 6720 may detect a touch gesture on the user's touch screen and transmit information about the touch gesture to the controller 6300 . The touch screen module 6720 according to some embodiments may recognize and analyze a touch code. The touch screen module 6720 may be configured as separate hardware including a controller.

Various sensors may be provided inside or near the touch screen to detect a touch or a proximity touch of the touch screen. As an example of a sensor for detecting a touch of a touch screen, there is a tactile sensor. A tactile sensor refers to a sensor that senses a touch of a specific object to the extent or higher than that felt by a human. The tactile sensor may sense various information such as the roughness of the contact surface, the hardness of the contact object, and the temperature of the contact point.

In addition, as an example of a sensor for detecting a touch of a touch screen, there is a proximity sensor.

The proximity sensor refers to a sensor that detects the presence or absence of an object approaching a predetermined detection surface or an object existing in the vicinity without mechanical contact using the force of an electromagnetic field or infrared rays. Examples of the proximity sensor include a transmission type photoelectric sensor, a direct reflection type photoelectric sensor, a mirror reflection type photoelectric sensor, a high frequency oscillation type proximity sensor, a capacitive type proximity sensor, a magnetic type proximity sensor, and an infrared proximity sensor. The user's touch gesture may include tap, touch & hold, double tap, drag, pan, flick, drag and drop, swipe, and the like.

The notification module 6730 may generate a signal for notifying the occurrence of an event in the device 2000 . Examples of events generated in the device 2000 include call signal reception, message reception, key signal input, schedule notification, and the like. The notification module 6730 may output a notification signal in the form of a video signal through the display unit 6210 , may output a notification signal in the form of an audio signal through the sound output unit 6220 , and a vibration motor 6230 . It is also possible to output a notification signal in the form of a vibration signal through

Some embodiments may also be implemented in the form of a recording medium containing instructions executable by a computer, such as program modules to be executed by a computer. Computer-readable media can be any available media that can be accessed by a computer and includes both volatile and nonvolatile media, removable and non-removable media. In addition, computer-readable media may include both computer storage media and communication media. Computer storage media includes both volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data. Communication media typically includes computer readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave, or other transport mechanism, and includes any information delivery media.

The description of the present invention described above is for illustration, and those of ordinary skill in the art to which the present invention pertains can understand that it can be easily modified into other specific forms without changing the technical spirit or essential features of the present invention. will be. Therefore, it should be understood that the embodiments described above are illustrative in all respects and not restrictive. For example, each component described as a single type may be implemented in a dispersed form, and likewise components described as distributed may be implemented in a combined form.

The scope of the present invention is indicated by the following claims rather than the above detailed description, and all changes or modifications derived from the meaning and scope of the claims and their equivalent concepts should be interpreted as being included in the scope of the present invention. do.

1000: wearable glasses
1010: frame
1020: speaker
1030: display
1040: user input unit
1050, 1060: Camera
1070: eye tracking camera
1100: sensing unit
1150: sensor
1200: processor
1300: communication department
1400: memory
1500: output
1600: power supply
2000: device

Claims (20)

In wearable glasses,
a display for displaying an image provided by the wearable glasses;
a sensing unit that obtains information about a wearing state in which a user is wearing the wearable glasses while the image is displayed on the display, and wherein the wearing state information includes an image obtained by photographing the user's eyes; and
An image displayed on the display based on the determined inclination of the wearable glasses for the user is determined based on at least one of a size and a shape of a reflective image formed when the image is reflected by the user's eyes. A wearable glass comprising a processor for adjusting the According to claim 1,
The sensing unit,
acquiring the wearing state information including information on a body part of the user who is wearing the wearable glasses,
The processor is
and determining a tilt of the wearable glasses with respect to the user by comparing the wearing state information with predetermined reference wearing state information. 3. The method of claim 2,
The sensing unit,
Obtaining the wearing state information including the image of the body part,
The processor is
The slope is determined by detecting a region corresponding to the body part from the image of the body part, acquiring a characteristic value from the detected region, and comparing the acquired characteristic value with a reference value included in the reference wearing state information. Characterized in determining, wearable glasses. According to claim 1,
The processor is
Obtaining at least one of the length of the major axis, the length of the minor axis, the angle of the major axis, and the angle of the minor axis from the photographed eye image, comparing the obtained value with a predetermined reference value, and based on the comparison result , determining a vertical inclination of the wearable glasses for the user,
determining a left/right tilt and a front/rear tilt of the wearable glasses for the user based on at least one of a size and a shape of the reflected image;
Wearable glasses, characterized in that the image displayed on the display is adjusted based on the determined vertical tilt and the determined left and right tilt and front/rear tilt. According to claim 1,
The display displays a test image,
The sensing unit,
Obtaining the wearing state information including an image of the user's eyes reflected in the test image,
The processor is
A region corresponding to the user's eye is detected from the photographed eye image, a reflection image of the test image is extracted from within the region corresponding to the eye, and at least one of a size and a shape of the reflection image is determined based on a predetermined criterion. A wearable glass that compares the wearing state information and determines the inclination based on the comparison result. According to claim 1,
The sensing unit,
The wearable glasses further measure a state value indicating a movement state of the wearable glasses, and obtain the wearing state information when the state value is greater than or equal to a predetermined value. According to claim 1,
The processor is
determining whether the determined slope is greater than or equal to a predetermined value;
The display is
When the determined slope is less than a predetermined value, an image adjusted based on the determined slope is displayed,
When the determined inclination is equal to or greater than a predetermined value, a notification image for notifying a user to rewrite the wearable glasses is displayed. A method for displaying an image by wearable glasses, the method comprising:
displaying an image provided by the wearable glasses on a display;
obtaining information on a state in which a user is wearing the wearable glasses;
determining a tilt of the wearable glasses with respect to the user by using the wearing state information; and
Comprising the step of adjusting the image displayed on the display based on the determined slope,
The wearing state information includes an image of the user's eyes,
The step of determining the inclination of the wearable glasses,
and determining an inclination of the wearable glasses with respect to the user based on at least one of a size and a shape of a reflection image formed when the image is reflected by the user's eyes. 9. The method of claim 8,
The step of obtaining the wearing state information,
Comprising the step of obtaining the wearing state information including information on the body part of the user wearing the wearable glasses,
The step of determining the inclination of the wearable glasses with respect to the user by using the wearing state information,
The method further comprising the step of determining an inclination of the wearable glasses for the user by comparing the wearing state information with predetermined reference wearing state information. 10. The method of claim 9,
The step of obtaining the wearing state information,
Comprising the step of obtaining the wearing state information including the image of the body part,
The step of determining the inclination of the wearable glasses with respect to the user by using the wearing state information,
detecting a region corresponding to the body part from the image of the body part;
obtaining a characteristic value from the detected region; and
The method further comprising the step of determining the inclination by comparing the acquired characteristic value with a reference value included in the reference wearing state information. 9. The method of claim 8,
The step of determining the inclination of the wearable glasses,
determining, based on at least one of a size and a shape of the reflected image, a left and right tilt and a front and back tilt of the wearable glasses for the user;
The step of determining the inclination of the wearable glasses,
obtaining at least one of a length of a major axis, a length of a minor axis, an angle of a major axis, and an angle of a minor axis of the eye from the photographed eye image;
comparing the obtained value with a predetermined reference value; and
Based on the comparison result, the method further comprising the step of determining a vertical inclination of the wearable glasses for the user. 9. The method of claim 8,
Displaying the image comprises:
displaying a test image;
The step of obtaining the wearing state information,
Comprising the step of acquiring the wearing state information including an image of the user's eyes reflected in the test image,
The step of determining the inclination of the wearable glasses with respect to the user by using the wearing state information,
detecting an area corresponding to the user's eye from the photographed eye image;
extracting a reflection image of the test image within an area corresponding to the eye;
comparing at least one of a size and a shape of the reflected image with predetermined reference wearing state information; and
The method further comprising determining the slope based on a result of the comparison. 9. The method of claim 8,
The step of obtaining the wearing state information,
measuring a state value indicating a movement state of the wearable glasses; and
When the state value is equal to or greater than a predetermined value, obtaining the wearing state information. 9. The method of claim 8,
Adjusting the image displayed on the display based on the determined inclination comprises:
determining whether the determined slope is equal to or greater than a predetermined value;
displaying an image notifying a user to rewrite the wearable glasses when the determined inclination is greater than or equal to a predetermined value; and
When the determined inclination is less than a predetermined value, adjusting the image displayed on the display based on the determined inclination. A computer-readable recording medium in which one or more programs including instructions for executing a method of displaying an image by wearable glasses are recorded,
The method is
displaying an image provided by the wearable glasses;
obtaining information on a state in which a user is wearing the wearable glasses;
determining a tilt of the wearable glasses with respect to the user by using the wearing state information; and
Displaying an image adjusted based on the determined inclination,
The wearing state information includes an image of the user's eyes,
The step of determining the inclination of the wearable glasses,
and determining an inclination of the wearable glasses with respect to the user based on at least one of a size and a shape of a reflection image formed when the image is reflected by the user's eyes. A device for displaying an image through wearable glasses, the device comprising:
a communication unit configured to receive, from the wearable glasses, information on a wearing state in which the user is wearing the wearable glasses, obtained from the wearable glasses; and
a controller for determining a tilt of the wearable glasses for the user by using the received wearing state information, and adjusting an image displayed through the wearable glasses based on the determined inclination;
The communication unit transmits the adjusted image to the wearable glasses,
The adjusted image is displayed through the wearable glasses,
The wearing state information includes an image of the user's eyes,
The controller is configured to determine a tilt of the wearable glasses for the user based on at least one of a size and a shape of a reflective image formed when the image is reflected by the user's eyes. A method for a device to display an image through wearable glasses, the method comprising:
receiving, from the wearable glasses, information on a wearing state in which a user is wearing the wearable glasses, obtained from the wearable glasses;
determining a tilt of the wearable glasses with respect to the user by using the received wearing state information; and
providing an image adjusted based on the determined inclination to the wearable glasses,
The adjusted image is displayed through the wearable glasses,
The wearing state information includes an image of the user's eyes,
The step of determining the inclination of the wearable glasses,
and determining an inclination of the wearable glasses with respect to the user based on at least one of a size and a shape of a reflection image formed when the image is reflected by the user's eyes. In wearable glasses,
a display for displaying an image to a user;
a sensing unit configured to obtain an image obtained by photographing the eyes of the user wearing the wearable glasses while the image is displayed on the display, and obtain information about the user's gesture; and
determining the inclination of the wearable glasses for the user based on at least one of a size and a shape of a reflective image formed when the image displayed on the display is reflected by the user's eyes,
Adjust the image displayed on the display based on the determined inclination,
and a processor to determine an adjustment value based on the information about the user's gesture, and further adjust the image displayed on the display based on the determined adjustment value. A method for displaying an image by wearable glasses, the method comprising:
displaying an image provided by the wearable glasses on a display;
acquiring an image obtained by photographing the eyes of a user wearing the wearable glasses while the image is displayed on the display;
determining an inclination of the wearable glasses with respect to the user based on at least one of a size and a shape of a reflective image formed when the image displayed on the display is reflected by the user's eyes;
adjusting the image displayed on the display based on the determined inclination;
obtaining information about the user's gesture;
determining an adjustment value based on the information on the user's gesture; and
and adjusting the image displayed on the display based on the determined adjustment value. A device for displaying an image through wearable glasses, the device comprising:
a communication unit configured to receive, from the wearable glasses, an image obtained by photographing the eyes of a user wearing the wearable glasses while the image is displayed on the display of the wearable glasses; and
The inclination of the wearable glasses is determined based on at least one of a size and a shape of a reflective image formed when the image displayed on the display of the wearable glasses is reflected by the user's eyes, and based on the determined inclination, the inclination of the wearable glasses is determined. A control unit for adjusting an image displayed through wearable glasses,
The communication unit transmits the adjusted image to the wearable glasses,
receiving information about a user's gesture obtained from the wearable glasses from the wearable glasses;
The control unit is
The device determines an adjustment value based on information about the user's gesture, and further adjusts an image displayed through the wearable glasses based on the determined adjustment value.

Images