TWI870744B - Control device and control method - Google Patents

Abstract

A control device is provided. The control device is adapted to control an object in a virtual world. The control device includes a display and a controller. The display is configured to display the virtual world. The controller is coupled to the display. The controller is configured to perform the following functions. In the virtual world, a control surface is formed around a user. A first ray is emitted from the object. Based on the first ray, a first control point is formed on the control surface. According to the first control point, a first control is performed on the object.

Description

Control device and control method

The present invention relates to a control device, and more particularly to a control device and a control method.

In order to bring immersive experiences to users, various technologies are constantly being developed, such as augmented reality (AR) and virtual reality (VR). AR technology allows users to bring virtual elements into the real world. VR technology allows users to enter a whole new virtual world and experience a different life. In addition, wearable devices are often used to provide these immersive experiences.

The present invention provides a control device and a control method, which enable a user to accurately control an object in a virtual world.

The control device of the present invention is suitable for controlling an object in a virtual world. The control device includes a display and a controller. The display is used to display the virtual world. The controller is coupled to the display. The controller is used to perform the following functions. In the virtual world, a control surface is formed around the user. A first ray is emitted from the object. Based on the first ray, a first control point is formed on the control surface. According to the first control point, a first control is performed on the object.

The control method of the present invention is suitable for controlling an object in a virtual world. The control method comprises: forming a control surface around a user; emitting a first ray from the object; forming a first control point on the control surface based on the first ray; and controlling the object according to the first control point.

Based on the above, by emitting rays from an object, the user can precisely control the object in the virtual world.

In order to make the content of the present invention more clearly understood, the following embodiments are specifically cited as examples by which the present invention can be truly implemented. In addition, wherever possible, the same reference numerals are used in the drawings and embodiments to represent the same or similar components.

Furthermore, unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by a person of ordinary skill in the art to which the present invention belongs. It will be further understood that those terms as defined in commonly used dictionaries should be interpreted as having a meaning consistent with their meaning in the context of the relevant art and the present invention, and will not be interpreted as an idealized or overly formal meaning unless expressly defined as such herein.

The present invention can be understood by referring to the following detailed description in conjunction with the attached drawings. It should be noted that, in order to facilitate the reader's understanding and for the sake of simplicity of the drawings, the various drawings in the present invention only depict a portion of the electronic device, and the specific components in the drawings are not drawn according to the actual scale. In addition, the number and size of each component in the drawings are only for illustration and are not used to limit the scope of the present invention.

It should be noted that the following embodiments may replace, reorganize, or mix the technical features of several different embodiments to complete other embodiments without departing from the spirit of the present invention. In addition, in the following specification and patent application, the words "including" and "comprising" are open-ended words, and therefore should be interpreted as "including but not limited to..."

In order to bring immersive experiences to users, various technologies are constantly being developed, such as augmented reality (AR) and virtual reality (VR). AR technology allows users to bring virtual elements into the real world. VR technology allows users to enter a whole new virtual world and experience a different life. In addition, wearable devices are often used to provide these immersive experiences.

In the virtual world, if you want to control an object, you can emit a ray from the control device or the user's position, such as any part of the user's body (wrist, finger). And the object pointed by the ray is the controlled object. However, this control method must first specify and identify the starting position of the ray and accurately judge the direction pointed by the user. The recognition accuracy is high, which increases the complexity of designing the control device. In addition, the user often needs to maintain a specific posture or gesture (such as clenching a fist or opening it) to perform correct control. In other words, when the direction pointed by the user is unclear or the user does not maintain a specific posture or gesture, misjudgment may occur. Furthermore, since the user must maintain a specific gesture, he cannot place it at will, which reduces the user experience. In addition, when moving the ray through a control device or gesture, it is easy for the control device or gesture to be misidentified, and it is easy for the ray to accidentally touch other objects when scanning. Therefore, how to accurately control objects in the virtual world in a simple and effective way while keeping the user in a comfortable posture has always been a topic pursued by technical personnel in this field.

FIG1A is a schematic diagram of a control device according to an embodiment of the present invention. Referring to FIG1A , the control device 100 is suitable for controlling an object in a virtual world. In addition, the control device 100 includes a display 120 and a controller 110. The display 120 is used to display the virtual world. The controller 110 is coupled to the display 120.

It is worth noting that the controller 110 can be used to perform the following functions. First, a control surface is formed around the user in the virtual world displayed by the display 120. Then, in the virtual world displayed by the display 120, a first ray is emitted from an object. In one embodiment, the first ray can be emitted from the surface of the object, and the first ray is a straight line or a curve, and the present invention is not limited. Then, in the virtual world displayed by the display 120, a first control point is formed on the control surface based on the first ray. In the virtual world displayed by the display 120, a first control is performed on the object according to the first control point. In this way, the user can simply and intuitively perform precise control of the object through the first control point on the control surface.

In one embodiment, the controller 110 may form the first control point on the control surface based on the intersection of the first ray emitted from the object and the control surface around the user. However, the disclosure is not limited thereto. In one embodiment, the control device 100 is, for example, a head-mounted display (HMD), wearable glasses (e.g., AR/VR goggles), an electronic device, other similar devices, or a combination of these devices. However, the disclosure is not limited thereto.

In one embodiment, the controller 110 is, for example, a microcontroller unit (MCU), a central processing unit (CPU), a microprocessor (Microprocessor), a digital signal processor (DSP), a programmable controller, a programmable logic device (PLD) or other similar devices or a combination of these devices, and the present invention is not limited thereto. In addition, in one embodiment, each function of the controller 110 can be implemented as a plurality of program codes. These program codes are stored in a memory and executed by the controller 110. Alternatively, in one embodiment, each function of the controller 110 can be implemented as one or more circuits. The present invention is not limited to implementing each function of the controller 110 by software or hardware.

In one embodiment, the display 120 includes, for example, an organic light-emitting diode (OLED) display device, a mini LED (Mini LED) display device, a micro LED (Micro LED) display device, a quantum dot (QD) LED display device, a liquid crystal display (Liquid-Crystal Display, LCD) device, a tiled display device, a foldable display device, or an electronic paper display (EPD). However, the present disclosure is not limited thereto.

Fig. 1B is a schematic diagram of a control system according to an embodiment of the present invention. The control system 10 includes a control device 100 and a camera 130. The details of the control device 100 can be found in the description of Fig. 1A and will not be elaborated here.

In one embodiment, the camera 130 includes, for example, a complementary metal oxide semiconductor (CMOS) camera or a charge coupled device (CCD) camera, and the auxiliary light unit includes an infrared illumination unit. However, the present disclosure is not limited to this. In addition, in one embodiment, the camera 130 may be disposed outside the control device 100, thereby forming an integrated control system 10. In another embodiment, the camera 130 may be disposed in the control device 100. That is, the present disclosure does not limit the location where the camera 130 is disposed.

In one embodiment, the camera 130 may be used to obtain (shoot) an image of the user in the real world. Furthermore, the controller 110 may perform image processing based on the obtained image to determine the user's intention. In other words, the controller 110 may generate a user command related to the user's intention based on the image obtained by the camera 130.

In one embodiment, the camera 130 may obtain a user image. The user image includes the user's hand (also referred to as a real hand) or a handheld remote control in the real world. The controller 110 may receive the above-mentioned user image from the camera 130. In addition, the controller 110 may determine whether the user's virtual hand or control anchor in the virtual world contacts the first control point on the control surface based on the user image. For example, the user's virtual hand or control anchor in the virtual world may correspond to the hand or handheld remote control in the real world, thereby performing operations in the virtual world. When the user's virtual hand or control anchor in the virtual world contacts the first control point, the user may perform various controls on the object through the first control point. In other words, in response to the virtual hand or the control anchor contacting the first control point, the controller 110 can perform the first control on the object. It should be noted that for the convenience of explanation, the following description is made with a virtual hand, but the present disclosure is not limited to the virtual hand, the control anchor or other objects with similar functions for manipulation in the virtual world.

In one embodiment, when the user's virtual hand approaches or touches the first control point, the first control point may be displayed as a highlighted (to be selected) state. In one embodiment, when the user's virtual hand approaches or touches the first control point, the surrounding image of the object corresponding to the first control point or the first ray emitted by the object may display a prompt mark, such as image color change, brightness increase, outline enhancement, display of text or animation description, playing of sound effects, or changes in the first ray color change, highlighting, jumping, etc. In this way, the user can clearly know that the object can be controlled via the first control point.

In one embodiment, the control surface may be, for example, a plane, a cubic surface, a cylindrical surface, a spherical surface, or an elliptical spherical surface, but the disclosure is not limited thereto. In one embodiment, the control surface distance between the control surface and the user may be a preset distance, such as 40 cm, but the disclosure is not limited thereto. In one embodiment, the control surface distance may be determined based on the user's personal parameters. In one embodiment, the user's personal parameters may include, for example, the user's arm length, forearm length, height, head length, or other various body data, but the disclosure is not limited thereto. Since human body proportions have certain regularities, the control surface distance that best suits the user may be calculated based on any one or a combination of multiple body data. In one embodiment, the center (e.g., center of gravity, center of a circle, or center of a sphere) or focus of a cubic surface, a cylindrical surface, a spherical surface, or an elliptical spherical surface may overlap with the user. For example, the control surface is a spherical surface. Moreover, the center of the spherical surface overlaps with the center point of the user, and the radius of the spherical surface is less than or equal to the length of the user's arm or forearm. Alternatively, the control surface may be two spherical surfaces. Moreover, the centers of the two spherical surfaces are located at the center point of the user's left shoulder and the center point of the user's right shoulder, respectively, and the radius of the two spherical surfaces is less than or equal to the length of the user's arm or forearm. In other words, the control surface may be formed around the user so that the user can easily touch the control surface. In addition, the user can choose a suitable control surface shape according to his or her own posture or application scenario. Furthermore, the user can also adjust the position and range (size) of the control surface according to the actual usage scenario, and the implementation method will be described in detail in the subsequent embodiments.

In one embodiment, a sensor is provided in the control device 100, and the sensor can detect the position where the user wears the control device 100, thereby automatically obtaining the user's personal parameters. In another embodiment, the camera 130 can shoot a posture image including the user's posture, and the controller 110 can automatically calculate the user's personal parameters based on the posture image. In another embodiment, the user can input the user's personal parameters to the control device 100 by himself. In other words, the present disclosure does not limit the method of obtaining the user's personal parameters.

It is worth mentioning that users control objects through control points on the adjacent control surface, rather than directly controlling distant objects. In this way, users can easily and accurately control objects, thereby increasing the user experience.

FIG. 2 is a schematic diagram of a usage scenario according to an embodiment of the present invention. Referring to FIG. 1A to FIG. 2 , scenario 200 includes a user U, an object 220 (also referred to as a cat), and an object 230 (also referred to as a button area) in a virtual world. Objects 220 and 230 may each include a plurality of control areas. Furthermore, each control area of object 220 and object 230 may respectively emit rays 221, 222, 223, 231, 232, 233 in the direction of user U. In addition, a control surface 210 is formed around user U.

In one embodiment, the control surface 210 is a virtual surface, and/or any of the rays 221, 222, 223, 231, 232, 233 is a virtual ray. That is, the user U may not normally see the existence of the control surface 210 and the rays 221, 222, 223, 231, 232, 233. Only when the hand of the user U approaches or touches the hidden control surface 210 or the control point, the control surface 210 or the rays 221, 222, 223, 231, 232, 233 will emerge. Moreover, when the hand of the user U moves away from the control surface 210 or the control point, the control surface 210 or the rays 221, 222, 223, 231, 232, 233 may enter a hidden state again. In other words, the controller 110 can determine whether to display or hide the control surface 210 or the rays 221, 222, 223, 231, 232, 233 according to the hand distance between the virtual hand of the user U and the control surface 210 or the hand distance between the virtual hand of the user U and the control point in the virtual world. In this way, the control surface 210 or the rays 221, 222, 223, 231, 232, 233 will only appear when the user U needs to control the object, so as not to affect the visual perception of the user U. In one embodiment, the control surface 210 and the rays 221, 222, 223, 231, 232, 233 may be replaced by the image effect of the object itself or the sound effect. In other words, the present disclosure does not limit the actual presentation method of the control surface 210 and the rays 221, 222, 223, 231, 232, 233, and FIG. 2 is only an implementation method for convenience of explanation.

It is worth mentioning that when the control surface 210 and the rays 221, 222, 223, 231, 232, 233 are in a hidden state, the virtual hand of the user U can be placed arbitrarily. In other words, since the control surface 210 and the rays 221, 222, 223, 231, 232, 233 are in a hidden state at this time, the virtual hand of the user U will not trigger the control of the control point on the control surface 210. In this way, when control is not required, the virtual hand of the user U can be placed comfortably without restriction, thereby increasing the use experience.

In one embodiment, the object 220 (cat) may include a plurality of control regions. For example, the head of the object 220 is the first control region, the body of the object 220 is the second control region, and the feet of the object 220 are the third control region. The first control region may emit a ray 221 (also referred to as the first ray), the second control region may emit a ray 222 (also referred to as the second ray), and the third control region may emit a ray 223 (also referred to as the third ray). In one embodiment, the starting points of the rays 221, 222, 223 may be located at the center points of the first control region, the second control region, and the third control region, respectively, but the disclosure is not limited thereto. Furthermore, the rays 221, 222, 223 may intersect with the control surface 210, respectively, thereby forming a first control point, a second control point, and a third control point.

It should be noted that in some embodiments, the user U may click on the first control point to make the object 220 produce the effect of being patted on the head. Furthermore, the user U may click on the second control point to make the object 220 produce the effect of being tickled. Furthermore, the user U may click on the third control point to make the object 220 move or rotate. When the object 220 moves or rotates, the multiple control points corresponding to the object 220 on the control surface 210 will also move accordingly. In other words, the controller 110 may perform different controls on the object 220 according to the different control points corresponding to the object 220. Furthermore, the controller 110 may move the position of the control point corresponding to the object 220 on the control surface 210 according to the movement or rotation of the object 220. In this way, the user U can perform various controls on the object 220 according to the multiple control points corresponding to the object 220.

In one embodiment, the object 230 (button area) may include a plurality of control areas. For example, the object 230 may include a first button, a second button, and a third button from top to bottom. The first button may emit a ray 231 (also referred to as a first ray), the second button may emit a ray 232 (also referred to as a second ray), and the third button may emit a ray 233 (also referred to as a third ray). The rays 231, 232, 233 may intersect with the control surface 210, respectively, to form a first control point, a second control point, and a third control point.

It should be noted that in some embodiments, the first control point, the second control point, and the third control point corresponding to the object 230 can be formed into a control group so that the first control point, the second control point, and the third control point can be adjusted synchronously (for example, adjusting the display method, the control point distance between the control points, or the control point position of an individual control point) to facilitate user selection and control.

For example, the control points in the control group can be displayed or hidden synchronously. For example, when the control group is displayed and the user's virtual hand approaches, touches or selects the control group, the controller 110 can fade the content other than the control group on the control surface 210. In other words, when the control group is selected, the control surface 210 can only display a single or multiple selected control groups to facilitate the user U to focus on the content of the control group.

For another example, the control group can be synchronously set to a magnification (equivalent to setting the control point distance). For example, when the control group is selected, the controller 110 can synchronously magnify the control point distance in the control group to make the distribution of the control points more dispersed than before, for example, the control point distance can be at least 5 cm, but the present disclosure is not limited to this. It should be noted that the adjustment method of the control point distance of the control point will be further described in detail in the subsequent embodiments.

For another example, the control group can be synchronously set with the offset position (control point position) of the control point. For example, when the control group is selected, the controller 110 can synchronously move the control points in the control group to a wider or fixed area on the control surface 210. In this way, by adjusting the control point positions of the control points in the control group, the user U can control the control points accurately.

For another example, the control points in the control group can be displayed on the control surface 210 in the form of a keyboard, so that the user can easily touch the keys on the keyboard to control the buttons on the object 230. In this way, the user U can intuitively input various commands (control commands) to the object 230, thereby increasing the convenience of manipulation in the virtual world and improving the user's input speed.

FIG. 3A is a schematic diagram of a manipulation gesture according to an embodiment of the present invention. FIG. 3B is a schematic diagram of a manipulation gesture according to an embodiment of the present invention. FIG. 3C is a schematic diagram of a manipulation gesture according to an embodiment of the present invention. Referring to FIG. 1A to FIG. 3C , gestures 300A, 300B, and 300C respectively illustrate various postures of the virtual hand H of the user U in the virtual world when performing control.

In one embodiment, the camera 130 may be used to capture an image of the user U's real hand gesture in the real world, and the controller 110 may perform image processing based on the image captured by the camera 130. Then, the controller 110 may transmit a signal to the display 120 so that the user U's virtual hand H in the virtual world generates a motion corresponding to the real hand in the real world. However, the present disclosure is not limited thereto.

The gestures 300A, 300B, and 300C each include a finger F1 (also called a first finger or thumb), a finger F2 (also called a second finger or index finger), and a pinch point PP. In one embodiment, the pinch point PP can be set at a position closer to the finger F1 in the line connecting the finger F1 to the finger F2. Generally speaking, when the virtual hand H of the user U pinches, the movement amount of the finger F1 is smaller than the movement amount of the finger F2. In this way, the user U can more easily complete the pinching of the pinch point PP.

In one embodiment, the user U can select or confirm the object 220 or the object 230 in the virtual world by pinching the pinch point PP. For example, when the pinch point PP overlaps or is the only control point closest to the corresponding object 220, the user U can pinch the pinch point PP to select or confirm the object 220. In one embodiment, the user U can pinch the pinch point PP for a preset time or release it after pinching to make a sub-window function appear or to confirm. In other words, the user U can pinch the pinch point PP to trigger the control point corresponding to the object 220, thereby controlling the object 220.

It is worth mentioning that when the user U is pinching the pinch point PP, the fingers other than the fingers F1 and F2 can maintain any posture. That is to say, the user U can arbitrarily choose a posture that he feels comfortable with (such as FIG. 3A or FIG. 3B or other gestures) and perform the pinching action. Moreover, in one embodiment, as shown in FIG. 3C , after the fingers F1 and F2 of the user U touch the pinch point PP, the fingers F1 and F2 can continue to be gathered toward the palm and maintain a fist posture. Moreover, by maintaining a fist posture, the user U can maintain the selected state of the pinch point PP. However, the present disclosure is not limited to this. For example, the user can use other gestures, such as sliding the hand in any direction up, down, left, right, forward, or backward, to select, trigger, or cancel the selection of the control point corresponding to the object 220. In this way, the user U can control the object 220 in the virtual world without being restricted by gestures, and can control the object in the most comfortable posture, thereby improving the user experience.

FIG4 is a schematic diagram of a control scenario according to an embodiment of the present invention. Referring to FIG1A to FIG4 , scenario 400 shows a user U's virtual hand H pinching and dragging in a virtual world. In one embodiment, after the user U pinches, the finger F1 and the finger F2 of the virtual hand H can remain in the pinching state to drag the controlled object (e.g., object 220).

It should be noted that, as shown in FIG4 , when the user U is dragging, the wrist of the virtual hand H can be turned arbitrarily. In other words, the wrist of the virtual hand H does not need to be maintained at a specific angle. If the wrist is maintained at a specific angle during dragging, the posture of the user U will be restricted. In other words, by only judging the relative relationship (such as distance) between the finger F1 and the finger F2 to maintain the pinching state, the user U can operate more flexibly. In this way, the user U can perform various operations in an ergonomic manner, thereby improving the user experience.

FIG. 5A is a schematic diagram of a control scenario according to an embodiment of the present invention. Referring to FIG. 1A to FIG. 5A , scenario 500A schematically illustrates the relationship between the user U, the control surface CTR, and the object OBJ. As shown in FIG. 5A , the object OBJ may include a control area E1 (also referred to as the first control area), a control area E2 (also referred to as the second control area), a control area E3 (also referred to as the third control area), and a control area E4 (also referred to as the fourth control area). The control areas E1~E4 may be located on the surface of the object OBJ, but the present disclosure is not limited thereto. Multiple rays (e.g., the first ray to the fourth ray) may be emitted from the control areas E1~E4 in the direction of the user U, respectively. In one embodiment, the starting points of the multiple rays may be located at the center points of the control areas E1~E4, respectively, but the present disclosure is not limited thereto. Furthermore, multiple rays corresponding to the control areas E1~E4 can intersect at the control plane CTR respectively, thereby forming the control area E1' (also called the first control point), the control area E2' (also called the second control point), the control area E3' (also called the third control point) and the control area E4' (also called the fourth control point).

Furthermore, the distance between the control areas E1-E4 may be referred to as the control area distance, and the distance between the control points E1'-E4' may be referred to as the control point distance. For example, the distance between the control area E1 and the control area E2 may be indicated as the control area distance D1A, and the distance between the control point E1' and the control point E2' may be indicated as the control point distance D2A.

In addition, the multiple rays corresponding to the control areas E1 to E4 may all intersect at the projection point P. For example, the rays emitted by the control area E1 (also referred to as the first rays) and the rays emitted by the control area E2 (also referred to as the second rays) all intersect at the projection point P. In other words, the starting points of the multiple rays corresponding to the control areas E1 to E4 are the centers of the control areas E1 to E4, respectively, and the end points of the multiple rays corresponding to the control areas E1 to E4 are all the projection point P.

It is worth mentioning that the control points E1'~E4' are formed based on the intersection of the corresponding rays on the control surface, and the rays corresponding to the control points E1'~E4' are formed based on the center of the control area E1~E4 as the starting point and the projection point P as the end point. In other words, when the position of the projection point P changes, the control point distance between the control points E1'~E4' (for example, the control point distance D2A) will also change accordingly.

For example, when the projection point P is located behind the user and moves to the rear of the user (e.g., the left side of FIG. 5A ) (i.e., the distance from the user U is farther than the original projection point P), the control points E1’~E4’ will move closer to each other. In other words, the control point distance between the control points E1’~E4’ (e.g., the control point distance D2A) will decrease. On the contrary, when the projection point P is located behind the user and moves to the front of the user (e.g., the right side of FIG. 5A ) (i.e., the distance from the user U is closer than the original projection point P), the control points E1’~E4’ will move away from each other. In other words, the control point distance between the control points E1’~E4’ (e.g., the control point distance D2A) will increase. Therefore, in one embodiment, the user U can adjust the control point distance (i.e., the display magnification) between the control points E1'-E4' displayed on the control surface CTR by adjusting the position of the projection point P (i.e., the projection point distance between the user U and the projection point P). In other words, the controller 110 can determine the control point distance D2A between the control point E1' and the control point E2' based on the position of the projection point P and the user U.

In addition, since the control plane CTR adjacent to the user U is located between the projection point P and the object OBJ, the control point distance between the control points E1'-E4' (e.g., the control point distance D2A) is smaller than the control area distance between the control areas E1-E4 (e.g., the control area distance D1A). In other words, the controller 110 can set the control point distance D2A between the first control point and the second control point to be smaller than the control area distance D1A between the control area E1 and the second control area E2 in response to the user U being located between the projection point P and the object OBJ.

Incidentally, the position and range (size) of the control surface CTR can be further adjusted. For example, the control surface CTR of FIG. 5A may include an origin C and a restricted range, and the rays and corresponding control points outside the restricted range are neither displayed nor controlled. In one embodiment, the origin C may be set at the center of the control surface CTR, but the present disclosure is not limited thereto. In one embodiment, when the user U wants to adjust the position of the control surface CTR, the origin C of the control surface CTR may be dragged with a single (or double) virtual hand H. For example, the user U may move the control surface CTR closer to or farther from the user U. For example, when both hands of the user U extend forward at the same time, the position of the control surface CTR (including the origin C) may be moved forward of the user U (i.e., farther from the user U than the original control surface CTR). That is, the controller 110 may set the control surface distance between the control surface CTR and the user U in response to the position of the origin C.

For another example, the user U can move the control surface CTR to any direction (such as up, down, left, and right of the user U) whose distance from the user U remains unchanged. For example, when both hands of the user U move to the left at the same time, the position of the control surface CTR (including the origin C) can be moved to the left of the user U. For another example, when both hands move to different directions, the control surface CTR can be moved in the direction of the center of both hands. For example, when the left hand of the user U moves to the left of the user U and the right hand moves to the top of the user U, the control surface CTR can be moved to the upper left. In this way, the user U can move the position of the control surface CTR by dragging the position of the origin C of the control surface CTR according to the actual usage scenario, thereby adjusting the range of control points covered by the control surface CTR.

Furthermore, in one embodiment, when the user U wants to adjust the range size of the control surface CTR, the user can use one or both virtual hands H to scale the restricted range of the control surface CTR. For example, the user U can enlarge or reduce the range of the control surface CTR. For example, when the hands of the user U move away from each other, the restricted range of the control surface CTR can be expanded to include more control points (for example, to include other control points other than the control points E1'~E4' in FIG. 5A). That is, in response to the user U expanding the restricted range CTR of the control surface, the invalid control point originally located outside the restricted range can be moved into the restricted range, thereby setting the invalid control point as a valid control point.

For another example, when the hands of the user U move toward each other, the restricted range of the control surface CTR can be reduced to include fewer control points (for example, reduced to only control points E2' and E3'). In other words, in response to the user U reducing the restricted range of the control surface CTR, the valid control points originally located in the restricted range can be moved outside the restricted range, thereby setting the valid control points as invalid control points. In this way, the user U can adjust the position and range of the control surface CTR according to the actual usage scenario to select the control points on the control surface CTR more accurately and quickly.

FIG. 5B is a schematic diagram of a control scenario according to an embodiment of the present invention. Referring to FIG. 1A to FIG. 5B , scenario 500B schematically illustrates the relationship between the user U, the control surface CTR, and the object OBJ. Compared to FIG. 5A , in which the user U is disposed between the projection point P and the object OBJ, in FIG. 5B , the object OBJ is disposed between the user U and the projection point P. The distance between the control area E1 and the control area E2 can be indicated as the control area distance D1B, and the distance between the control point E1' and the control point E2' can be indicated as the control point distance D2B. In addition, FIG. 5B illustrates a restriction range R, which will be described in subsequent embodiments. For details about the user U, the control surface CTR, and the object OBJ, please refer to the description of FIG. 5A , which will not be elaborated here.

Similar to FIG. 5A , when the position of the projection point P of FIG. 5B changes, the control point distances between the control points E1’~E4’ (e.g., the control point distance D2B) will also change accordingly. For example, when the projection point P is located in front of the user and moves to the front of the user (e.g., the right side of FIG. 5B ) (i.e., the distance from the user U is farther than the original projection point P), the control points E1’~E4’ will move closer to each other. In other words, the control point distances between the control points E1’~E4’ (e.g., the control point distance D2B) will decrease. On the contrary, when the projection point P is located in front of the user and moves to the back of the user (e.g., the left side of FIG. 5B ) (i.e., the distance from the user U is closer than the original projection point P), the control points E1’~E4’ will move away from each other. In other words, the control point distance between the control points E1'~E4' (e.g., the control point distance D2B) will increase. Therefore, in one embodiment, the user U can adjust the control point distance between the control points E1'~E4' displayed on the control surface CTR (i.e., the display magnification) by adjusting the position of the projection point P (i.e., the projection point distance between the user U and the projection point P). In other words, the controller 110 can determine the control point distance D2B between the control point E1' and the control point E2' based on the position of the projection point P and the user U.

In addition, since the object OBJ is disposed between the projection point P and the control plane CTR adjacent to the user U, the control point distance between the control points E1'-E4' (e.g., the control point distance D2B) is greater than the control area distance between the control areas E1-E4 (e.g., the control area distance D1B). In other words, the controller 110 may set the control point distance D2B between the first control point and the second control point to be greater than the control area distance D1B between the control area E1 and the second control area E2 in response to the object OBJ being located between the projection point P and the user U.

In addition, in one embodiment, the projection point P can be set between the user U, the control surface CTR and the object OBJ to obtain control points that are oppositely distributed up and down and left and right corresponding to the control area on the object OBJ. Alternatively, in one embodiment, the projection point P can be set to infinity in front of or behind the user U to obtain control points that are uniformly distributed corresponding to the control area on the object OBJ, that is, the control point distance between each control point will be the same as the distance between each corresponding control area on the object OBJ. In this way, by adjusting the position of the projection point P and the projection point distance between the user U and the projection point P, the user U can adjust the display magnification (magnification) of the control surface CTR or any control group on the control surface CTR, that is, can adjust the control point distance on the control surface CTR (for example, the control point distance D2A in Figure 5A or the control point distance D2B in Figure 5B), thereby obtaining a better visual experience and being able to simply and effectively control objects accurately in the virtual world. Furthermore, by adjusting the position or range of the control surface CTR (for example, by adjusting the origin C of the control surface CTR), the user U can adjust the control surface distance between the user U and the control surface CTR and the range of control points covered by the control surface CTR (limited range R), thereby being able to manipulate the control surface CTR in a comfortable posture, thereby increasing a better user experience.

It should be noted that, for the convenience of explanation, the control surface CTR in FIG. 5B is shown as a finite length (finite plane). However, in some embodiments, the control surface CTR is a plane that extends infinitely. In addition, a restriction range R is shown in FIG. 5B to indicate the effective control area or effective display area of the control surface CTR. In one embodiment, the restriction range R is the coverage range of the effective control points on the control surface CTR. For example, the control points within the restriction range R are effective control points, and the control points outside the restriction range R are invalid control points. That is, in response to the virtual hand of the user U touching the effective control point within the control range R in the virtual world, the controller 110 can accept the control command of the user U. Furthermore, in response to the virtual hand of the user U touching an invalid control point outside the control range R in the virtual world, the control command of the user U is not accepted.

For example, FIG. 5B shows a virtual line (ray) starting from the projection point P and passing through the object OBJ, and intersecting outside the restricted range R of the control surface CTR, thereby forming an invalid control point. Since the invalid control point is outside the restricted range R of the control surface CTR, the controller 110 will not execute the operation of the user U on the invalid control point. In this way, by setting the restricted area R, the occurrence of erroneous operation can be avoided, thereby increasing the user experience.

FIG6 is a flow chart of a control method according to an embodiment of the present invention. Referring to FIG1 and FIG6 , the control method 600 schematically shows the operation flow of the control device 100, but the present disclosure is not limited thereto. In this embodiment, the control method 600 includes steps S610 to S640.

In step S610, the controller 110 may transmit a signal to the display 120, so that the control surface 210 is formed around the user U in the virtual world displayed by the display 120. In step S620, the controller 110 may transmit a signal to the display 120, so that an object (e.g., object 220 or object 230) in the virtual world displayed by the display 120 emits a first ray (e.g., one of rays 221-233). In step S630, the controller 110 may transmit a signal to the display 120, so that the display 120 may form (display) a first control point on the control surface 210 based on the first ray. In step S640, the controller 110 may transmit a signal to the display 120, so that the user U can control the object (e.g., the object 220 or the object 230) according to the first control point. It should be noted that the relevant implementation details can be referred to the description of FIG. 1 to FIG. 5B, and will not be elaborated here. In this way, the user U can simply and intuitively control the object accurately through the first control point on the control surface 210.

In summary, the control device of the present invention uses a control surface to control objects in the virtual world. Therefore, the user can maintain a comfortable posture while being able to simply and effectively control the objects in the virtual world accurately.

10: Control system 100: Control device 110: Controller 120: Display 130: Camera 200, 400, 500A, 500B: Context 210, CTR: Control surface 220, 230, OBJ: Object 221, 222, 223, 231, 232, 233: Rays 300A, 300B, 300C: Gestures 600: Control method C: Origin D1A, D1B: Control area distance D2A, D2B: Control point distance E1, E2, E3, E4: Control area E1’, E2’, E3’, E4’: Control point F1, F2: Fingers H: Virtual hand P: Projection point PP: Pinch point R: Restriction range S610, S620, S630, S640: Steps U: User

FIG. 1A is a schematic diagram of a control device according to an embodiment of the present invention. FIG. 1B is a schematic diagram of a control system according to an embodiment of the present invention. FIG. 2 is a schematic diagram of a use scenario according to an embodiment of the present invention. FIG. 3A is a schematic diagram of a control gesture according to an embodiment of the present invention. FIG. 3B is a schematic diagram of a control gesture according to an embodiment of the present invention. FIG. 3C is a schematic diagram of a control gesture according to an embodiment of the present invention. FIG. 4 is a schematic diagram of a control scenario according to an embodiment of the present invention. FIG. 5A is a schematic diagram of a control scenario according to an embodiment of the present invention. FIG. 5B is a schematic diagram of a control scenario according to an embodiment of the present invention. FIG. 6 is a flow chart of a control method according to an embodiment of the present invention.

100: Control device

110: Controller

120: Display

Claims (14)

A control device is suitable for controlling an object in a virtual world, wherein the control device comprises: a display for displaying the virtual world; and a controller coupled to the display, wherein the controller is used to: form a control surface around a user in the virtual world, wherein the distance between the object and the user is greater than the distance between the control surface and the user; emit a first ray from the object; form a first control point on the control surface based on the first ray; perform a first control on the object according to the first control point; and determine whether to hide the control surface or the first ray according to the hand distance between the virtual hand of the user and the control surface or the first control point in the virtual world. The control device as described in claim 1, wherein the controller is further used to: receive a user image, wherein the user image includes the user's hand or handheld remote control in the real world; determine whether the control anchor corresponding to the user's hand or the handheld remote control in the virtual world touches the first control point based on the user image; and perform the first control on the object in response to the control anchor touching the first control point. A control device as described in claim 1, wherein the controller is further used to: determine the control surface distance between the control surface and the user according to the personal parameters of the user. A control device as described in claim 1, wherein the controller is further used to: form the first control point on the control surface based on the intersection point of the first ray and the control surface. A control device as described in claim 1, wherein the controller is further used to: determine whether to display the control surface or the first ray according to the hand distance between the virtual hand of the user and the control surface or the first control point in the virtual world. A control device as described in claim 1, wherein the control surface includes a plane, a cubic surface, a cylindrical surface, a spherical surface, or an elliptical spherical surface. A control device as described in claim 1, wherein the object includes a first control area and a second control area, and the controller is further used to: emit the first ray from the first control area; emit the second ray from the second control area, wherein the first ray and the second ray intersect at a projection point; form a second control point on the control surface based on the second ray; and determine the control point distance between the first control point and the second control point based on the projection point and the position of the user. The control device as claimed in claim 7, wherein the controller is further used to: in response to the user being located between the projection point and the object, set the control point distance between the first control point and the second control point to be smaller than the control area distance between the first control area and the second control area; or in response to the object being located between the projection point and the user, set the control point distance between the first control point and the second control point to be larger than the control area distance between the first control area and the second control area. A control device as described in claim 1, wherein the object includes a first control area and a second control area, and the controller is further used to: emit the first ray from the first control area; emit the second ray from the second control area; form a second control point on the control surface based on the second ray; and form a control group with the first control point and the second control point to synchronously adjust the first control point and the second control point. A control device as described in claim 9, wherein the controller is further used to: in response to the control group being selected, fade out the content other than the control group on the control surface. The control device as claimed in claim 1, wherein the control surface includes a restriction range, and the controller is further used to: respond to the user's virtual hand touching a valid control point within the restriction range in the virtual world, accept the user's control instruction; and respond to the user's virtual hand touching an invalid control point outside the restriction range in the virtual world, not accept the user's control instruction. In the control device as claimed in claim 11, the controller is further used to: in response to the user expanding the restriction range and the invalid control point being moved into the restriction range, set the invalid control point as the valid control point; and in response to the user reducing the restriction range and the valid control point being moved outside the restriction range, set the valid control point as the invalid control point. A control method is suitable for controlling an object in a virtual world, wherein the control method comprises: forming a control surface around a user, wherein the distance between the object and the user is greater than the distance between the control surface and the user; emitting a first ray from the object; forming a first control point on the control surface based on the first ray; and controlling the object according to the first control point; and determining whether to hide the control surface or the first ray according to the hand distance between the virtual hand of the user and the control surface or the first control point in the virtual world. The control method as described in claim 13 further comprises: receiving a user image, wherein the user image includes the user's hand or handheld remote control in the real world; judging whether the control anchor corresponding to the user's hand or the handheld remote control in the virtual world touches the first control point according to the user image; and performing the first control on the object in response to the control anchor touching the first control point.

Images