KR20100074461A - 3d mouse apparatus and method that control using action sense - Google Patents

Abstract

Provided are a three-dimensional mouse device and method for controlling using motion detection.

The proposed 3D mouse apparatus and method for controlling using motion detection can provide a mouse motion by detecting a hand motion through a muscle movement of a wrist and mapping the detected hand motion to a mouse motion.

The proposed three-dimensional mouse apparatus and method for controlling by using motion detection can provide a mouse motion by detecting an arm motion indicating rotation and movement of an arm through an inertial sensor and mapping the detected arm motion to a mouse motion.

The proposed 3D mouse apparatus and method for controlling by using motion detection may provide a mouse motion corresponding to a combination of the two motions by using a wrist-band type sensor unit for detecting a hand motion and an arm motion.

Description

3D MOUSE APPARATUS AND METHOD THAT CONTROL USING ACTION SENSE}

The following description relates to a three-dimensional mouse device and method for controlling by using motion detection, and in particular, provides a mouse motion corresponding to a combination of the two motions by using a wrist-band type sensor unit for detecting a hand motion and an arm motion. A three-dimensional space mouse apparatus and method are described.

In general, the mouse, which is one of the peripheral devices used in the computer, controls the movement of the pointer displayed on the monitor screen. To move the pointer, the mouse located in a certain space must be moved up, down, left, and right.

The mouse is equipped with a click button and a drag button on the upper surface of the main body, and consists of a pointer using a ball or an optical element on the lower surface of the mouse. The drag is activated and the middle button is used to operate the computer. Here, the pointer refers to a pointing cursor that visually moves in accordance with the movement of a mouse in an operating system supporting a GUI.

However, when using such a computer mouse, the shape of the user's hand is positioned as if the palm is covered with the palm according to the structure of the mouse. Therefore, the wrist should be kept at a predetermined angle to the left side. It is necessary to continue working for a long time. Especially in the case of index finger, operation of left button and drag is concentrated, which causes pain to wrist, index ligament, upper muscle of arm, elbow, shoulder neck, and fatigue In the long term, there are disadvantages that occur in chronic occupational diseases according to each part. In addition, in the case of a wired mouse, there is a distance limitation using the mouse. In the case of a wireless mouse to solve this problem, movement is limited because it operates on a plane supporting the mouse.

In order to solve the problem of the mouse as described above, a spatial mouse has been proposed that detects and points the position in three-dimensional space and controls the movement of the mouse. In the case of such a spatial mouse, the pointing direction or position is mapped on the screen according to the hand and arm motion while the mouse device is directly held by the hand, and a separate button is provided to perform a click operation. In the conventional space mouse, since the mouse device is held in the hand, the movement of the hand is limited and natural hand movement is impossible (for example, it is difficult to straighten the hand) and the button is pressed. It is inconvenient to perform the operation.

The present invention relates to a three-dimensional mouse apparatus and method for controlling by using motion detection.

The present invention relates to a three-dimensional space mouse apparatus and method for providing a mouse movement corresponding to a combination of two movements by using a wrist-band type sensor unit for detecting a hand movement and an arm movement.

The three-dimensional mouse device according to the proposed embodiment, the sensor unit for detecting at least one of the hand motion according to the movement of the cuff muscles or the arm motion according to the up, down, left and right and rotational movement of the arm; And a controller configured to check and output a mouse motion corresponding to a combination of the hand motion and the arm motion detected by the hand motion sensor part and the arm motion sensor part.

In this case, the sensor unit, a hand motion sensor unit for detecting a hand motion by detecting the movement of the wrist muscle; And an arm motion sensor unit configured to detect an arm motion as up, down, left, right, and rotational movements of the arm.

At this time, the hand motion sensor unit is located in the wrist in a band shape to detect the movement of the wrist muscle using one or more Force Sensing Resistor (FSR) sensor.

In this case, the hand gesture sensor unit detects the hand gesture by using at least one of a force sensing resistor (FSR) sensor, a force sensor, or a strain guage sensor.

In this case, the arm motion sensor unit includes an inertial measurement unit (IMU) including at least one of an acceleration sensor, a gyro sensor, and a geomagnetic sensor.

In this case, the arm motion sensor unit detects the arm motion using at least one of an inertial measurement unit (IMU), an infrared (IR) sensor, or an ultrasonic sensor.

In this case, if the hand gesture is a fist, the controller outputs a mouse movement by clicking, and the arm movement is output by moving the mouse pointer.

In this case, the controller outputs a mouse motion of dragging the selected object when the hand motion is in a fist state and the arm motion is moving up, down, left, and right.

At this time, the control unit, when the arm movement is a movement below the predetermined threshold value in the palm facing up state and the hand movement is the movement of the palm up and down while the wrist is fixed, the mouse movement to turn the wheel of the mouse up Output

At this time, the control unit, when the arm movement is a movement below the predetermined threshold value in the state of the palm facing down and the hand movement is the movement of the palm up and down while the wrist is fixed, the mouse to turn the wheel of the mouse down Output by action.

At this time, the controller outputs a mouse operation for rotating the selected object in the same rotation direction, if the hand movement is a fist state and the arm movement is a rotation around the forearm.

The three-dimensional mouse control method according to the present embodiment, the step of detecting at least one of the hand motion according to the movement of the wrist muscle or the arm motion according to the up, down, left and right and rotational movement of the arm; Identifying a mouse motion corresponding to the detected combination of the hand motion and the arm motion; And outputting the confirmed mouse operation.

A three-dimensional mouse control method according to an embodiment of the present invention includes detecting a hand motion by detecting a movement of a wrist muscle through a hand motion sensor unit; Detecting an arm motion which is an up, down, left, right, and rotational movement of the arm through the arm motion sensor unit; Identifying a mouse motion corresponding to the detected combination of the hand motion and the arm motion; And outputting the confirmed mouse operation.

In this case, the hand motion is detected by checking the movement of the cuff muscles by using one or more Force Sensing Resistor (FSR) sensors positioned on the wrist in a band shape.

In this case, the hand gesture is sensed using at least one of a force sensing resistor (FSR) sensor force sensor or a strain guage sensor.

In this case, the arm motion is sensed using an inertial measurement unit (IMU) including at least one of an acceleration sensor, a gyro sensor, and a geomagnetic sensor.

In this case, the arm motion is detected using at least one of an inertial measurement unit (IMU), an infrared (IR) sensor, or an ultrasonic sensor.

In this case, the detecting of the mouse motion corresponding to the detected combination of the hand motion and the arm motion may include outputting a mouse motion by clicking the hand motion when the hand motion is a fist, and confirming the arm motion by moving the mouse pointer. .

In this case, the detecting of the mouse motion corresponding to the detected combination of the hand motion and the arm motion may be performed by dragging the selected object when the hand motion is in a fist-capped state and the arm motion is moved up, down, left, and right. Check it.

In this case, the detecting of the mouse motion corresponding to the detected combination of the hand motion and the arm motion may be performed when the arm motion is less than or equal to a predetermined threshold value with the palm facing upward and the hand motion fixed the wrist. If you move the palm up and down, you can check the mouse movement by turning the mouse wheel up.

In this case, the detecting of the mouse motion corresponding to the detected combination of the hand motion and the arm motion is a motion of the arm motion less than a preset threshold with the palm facing downward, and the hand motion fixing the wrist. If you move the palm up and down in the state, check the mouse movement by turning the mouse wheel down.

In this case, the detecting of the mouse motion corresponding to the detected combination of the hand motion and the arm motion may include: detecting the hand motion by the hand motion sensor part and the arm motion by the arm motion sensor part. If it is a rotation, the selected object is confirmed by a mouse motion that rotates in the same rotation direction.

The 3D mouse device and method for providing a mouse motion corresponding to a combination of the two motions by using a wrist band-type sensor unit for detecting a hand motion and an arm motion according to the proposed embodiment is a hand motion in a state where a hand can freely move in space. User interface can be implemented by classifying.

Hereinafter, exemplary embodiments will be described in detail with reference to the accompanying drawings. In the above-described exemplary embodiment, the mouse controlling the mouse pointer in the 3D space is called a 3D mouse or a space mouse.

1 is a diagram illustrating a configuration of a 3D mouse device controlled by using motion detection according to an exemplary embodiment. Referring to FIG. 1, the 3D mouse device includes a sensor unit 110 including a hand gesture sensor 112 and an arm gesture sensor 114, and a controller 110.

Hand motion sensor 112 is a sensor for detecting the motion of the wrist by detecting the movement of the wrist. The hand gesture sensor 112 may use a Force Sensing Resistor (FSR) sensor to detect muscle movement. The hand gesture sensor 112 may detect a hand gesture by placing a plurality of sensors on the inner surface of the wrist band type as shown in FIG. 2 in order to detect the movement of the arm muscle.

2 is a diagram illustrating a schematic appearance of a wrist band type 3D mouse device controlled by using motion detection according to an exemplary embodiment.

The hand motion sensor 112 may determine that the muscle contracted when the resistance value compared to the pressure measured by the FSR sensor is greater than or equal to a preset value, and detect the hand motion by contraction of the muscle detected by the plurality of FSR sensors. Representative hand motions detected by the hand motion sensor unit 112 may distinguish a fist state and a hand-open state, and may detect a movement of a wrist.

The hand motion sensor 112 is a sensor for detecting hand motion, and the hand motion may be detected using other sensors as well as an FSR sensor. Accordingly, the hand gesture sensor 112 may detect a hand gesture using a force sensor or a strain guage sensor that detects a force of the arm. When using a force sensor, the hand motion sensor 112 may detect a hand motion similarly to when using an FSR sensor.

When the hand motion sensor 112 uses a strain guage sensor to detect a hand motion, the hand motion sensor 112 may detect the movement of the hand by measuring the tension / shrinkage degree of the surface. For example, when the hand motion sensor 112 detects a hand motion by using a strain guage sensor in a three-dimensional mouse composed of a wrist-type band, the fist state and the hand-wound state are distinguished by the tension / contraction of the band. The degree of bending of the wrist can be measured in the direction of bending.

In addition, the hand gesture sensor 112 may use several sensors to measure hand gestures, and may use two or more of an FSR sensor, a force sensor, and a strain sensor.

The arm motion sensor unit 114 is a sensor for detecting arm motion that is up, down, left, right, and rotational movements of the arm. The arm motion sensor 114 may be configured as an inertial measurement unit (IMU) including at least one of an acceleration sensor, a gyro sensor, and a geomagnetic sensor.

Here, the acceleration sensor is a sensor for measuring the acceleration in each direction in the three-dimensional space, the gyro sensor is a sensor for detecting the rotation of each of the three-dimensional axis, the geomagnetic sensor is a sensor for detecting the earth's magnetism to display the orientation.

The controller 110 checks and outputs a mouse motion corresponding to the combination of the hand motion and the arm motion detected by the hand motion sensor 112 and the arm motion sensor 114.

The control unit 110 outputs a mouse action as a click when the hand gesture is a fist as a result of the detection of the hand gesture sensor 112, and outputs the mouse action as a movement of the mouse pointer as a result of the detection of the arm gesture sensor 114. .

If the result of the detection of the hand motion sensor 112 is the fist movement and the result of the detection of the arm motion sensor 114 is the movement of the arm up, down, left, and right, the controller 110 provides the selected object according to the movement of the arm motion. The output is output by mouse action.

The control unit 110 is a palm motion in a state in which the arm motion is less than the predetermined threshold value in the state where the palm is facing upward and the palm motion is fixed by the hand as the result of the detection of the hand motion sensor 112. If it moves up and down, the mouse wheel is rotated upward.

The control unit 110 is a motion of the arm motion sensor unit 114 as a result of the arm motion is below the preset threshold in the palm facing down, and the hand motion of the hand motion sensor unit 112 as a result of the hand motion is fixed to the wrist If the palm moves up and down, the mouse wheel is turned down to output the mouse motion.

The controller 110 rotates the selected object in the same rotational direction when the result of the sensing of the hand motion sensor 112 is the fist movement and the result of the sensing of the arm motion sensor 114 is the rotation around the forearm. Output mouse movements.

Meanwhile, the three-dimensional mouse device according to the proposed embodiment described with reference to FIG. 1 can detect a user's arm motion in a three-dimensional space, and thus the human arm motion is a virtual character in a virtual world (VW: Virtual World). It can also be used by mapping with the arm action of.

Hereinafter, a method of controlling a mouse motion using motion detection in the band-type three-dimensional mouse device configured as described above will be described with reference to the accompanying drawings.

3 is a flowchart illustrating a process of controlling a mouse movement using motion detection in a 3D mouse device according to an exemplary embodiment.

Referring to FIG. 3, when the 3D mouse device receives the spatial mouse control request in step 300, the 3D mouse device detects the hand motion by detecting the movement of the cuff muscle through the hand motion sensor in step 302, and in step 304. Proceeds to detect the arm motion of the upper and lower, left and right and rotational movement of the arm through the arm motion sensor. In this case, the order of steps 302 and 304 may be changed and may be performed simultaneously.

In operation 306, the 3D mouse apparatus checks the mouse motion corresponding to the combination of the detected hand motion and the arm motion, proceeds to step 308, outputs the confirmed mouse motion, and proceeds to step 310. Checks if mouse control is terminated. If it is confirmed that the control of the space mouse is not finished, the process returns to step 302 and repeats a series of steps up to step 310.

In operation 306, if the hand movement is a fist, the mouse movement is confirmed by clicking the mouse movement, and the arm movement is confirmed by the movement of the mouse pointer.

As a more detailed example, the mouse motion checked in step 306 may be confirmed by a mouse motion of dragging the selected object according to the movement of the arm motion when the hand motion is a fist and the arm motion moves up, down, left, and right.

As another example, the mouse motion checked in step 306 is a movement of the palm up and down when the arm motion is below a preset threshold with the palm facing upward, and the palm motion of the palm moving up and down with the wrist fixed. Confirm by mouse action.

As another example, the mouse motion checked in step 306 is the movement of the palm up and down when the arm motion is below a preset threshold with the palm facing downward and the palm motion of the palm moving up and down with the wrist fixed. This can be confirmed by moving the mouse down.

As another example, the mouse motion checked in step 306 may be confirmed as a mouse motion in which the selected object is rotated in the same rotational direction if the hand motion is a fist and the arm motion is a rotation about the forearm.

The methods described above may be embodied in the form of program instructions that may be executed by various computer means and may be recorded in a computer readable medium. The computer readable medium may include program instructions, data files, data structures, etc. alone or in combination. Program instructions recorded on the media may be those specially designed and constructed for the purposes of the present invention, or they may be of the kind well-known and available to those having skill in the computer software arts. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tape, optical media such as CD-ROMs, DVDs, and magnetic disks, such as floppy disks. Magneto-optical media, and hardware devices specifically configured to store and execute program instructions, such as ROM, RAM, flash memory, and the like. The medium may be a transmission medium such as an optical or metal line, a wave guide, or the like, including a carrier wave for transmitting a signal designating a program command, a data structure, or the like. Examples of program instructions include not only machine code generated by a compiler, but also high-level language code that can be executed by a computer using an interpreter or the like. The hardware device described above may be configured to operate as one or more software modules to perform the operations of the present invention, and vice versa.

As described above, the present invention has been described by way of limited embodiments and drawings, but the present invention is not limited to the above embodiments, and those skilled in the art to which the present invention pertains various modifications and variations from such descriptions. This is possible.

Therefore, the scope of the present invention should not be limited to the described embodiments, but should be determined not only by the claims below but also by the equivalents of the claims.

1 is a view showing the configuration of a three-dimensional mouse device to control using motion detection according to an embodiment of the present invention,

2 is a view illustrating a schematic appearance of a wrist band type 3D mouse device controlled using motion detection according to an embodiment of the present disclosure;

3 is a flowchart illustrating a process of controlling a mouse movement using motion detection in a 3D mouse device according to an exemplary embodiment.

Claims (21)

Sensor unit for detecting at least one of the hand motion according to the movement of the wrist muscle or the arm motion according to the up, down, left and right and rotational movement of the arm; And And a controller configured to check and output a mouse motion corresponding to the combination of the hand motion and the arm motion detected by the hand motion sensor part and the arm motion sensor part. The method of claim 1, wherein the sensor unit, Hand motion sensor unit for detecting a hand motion by detecting the movement of the wrist muscle; And A three-dimensional mouse device including an arm motion sensor for detecting an arm motion that is up, down, left and right and rotational movement of the arm. The method of claim 2, wherein the hand gesture sensor, 3D mouse device which is located on the wrist in a band shape to detect the movement of the cuff muscles using one or more Force Sensing Resistor (FSR) sensor. The method of claim 2, wherein the hand gesture sensor, Force Sensing Resistor (FSR) sensor A three-dimensional mouse device for detecting the hand motion using at least one of a force sensor and a strain guage sensor. The arm motion sensor of claim 2, A three-dimensional mouse device comprising an inertial measurement unit (IMU) including at least one of an acceleration sensor, a gyro sensor, and a geomagnetic sensor. The arm motion sensor of claim 2, 3D mouse device for detecting the arm movement using at least one of an inertial measurement unit (IMU), infrared (IR) sensor or ultrasonic sensor. The method of claim 1, wherein the control unit, 3. The mouse apparatus of claim 3, wherein when the hand gesture is a fist, a mouse motion is output as a click and the arm motion is output as a movement of a mouse pointer. The method of claim 1, wherein the control unit, 3. The mouse apparatus of claim 3, wherein when the hand gesture is in a clenched state and the arm gesture is moved up, down, left, or right, a mouse motion of dragging a selected object is output. The method of claim 1, wherein the control unit, 3D mouse device outputting a mouse motion of turning the wheel of the mouse upward when the arm motion is a motion below the preset threshold in the palm facing upward and the palm motion is moving the palm up and down while the wrist is fixed. . The method of claim 1, wherein the control unit, 3D outputting the mouse motion by turning the wheel of the mouse downward when the arm motion is a motion below the preset threshold value when the palm is facing downward and the hand motion is a motion of moving the palm up and down while the wrist is fixed. Mouse device. The method of claim 1, wherein the control unit, And outputting a mouse motion of rotating the selected object in the same rotation direction when the hand motion is in a fist state and the arm motion is a rotation about the forearm. Sensing at least one of a hand motion according to the movement of the wrist muscle or an arm motion according to the up, down, left, right, and rotational movement of the arm; Identifying a mouse motion corresponding to the detected combination of the hand motion and the arm motion; And And outputting the identified mouse movement. The method of claim 12, wherein the hand gesture, 3D mouse control method is detected by checking the movement of the cuff muscles using one or more Force Sensing Resistor (FSR) sensor located on the wrist in a band shape. The method of claim 12, wherein the hand gesture, Force Sensing Resistor (FSR) Sensor A three-dimensional mouse control method sensed using at least one of a force sensor or a strain guage sensor. The method of claim 12, wherein the arm movement, 3. A method of controlling a 3D mouse, which is detected using an inertial measurement unit (IMU) including at least one of an acceleration sensor, a gyro sensor, and a geomagnetic sensor. The method of claim 12, wherein the arm movement, A three-dimensional mouse control method sensed using at least one of an inertial measurement unit (IMU), an infrared (IR) sensor or an ultrasonic sensor. The method of claim 12, wherein identifying a mouse movement corresponding to the detected combination of the hand gesture and the arm gesture comprises: And outputting a mouse motion as a click when the hand motion is a fist, and confirming the arm motion as a movement of a mouse pointer. The method of claim 12, wherein identifying a mouse movement corresponding to the detected combination of the hand gesture and the arm gesture comprises: 3. The method of claim 3, wherein when the hand gesture is in a fist state and the arm gesture is moved up, down, left, and right, a mouse operation of dragging a selected object is confirmed. The method of claim 12, wherein identifying a mouse movement corresponding to the detected combination of the hand gesture and the arm gesture comprises: 3D mouse control to check if the arm movement is a movement below the preset threshold in the palm facing up and the palm movement up and down while holding the wrist is fixed, by turning the mouse wheel up Way. The method of claim 12, wherein identifying a mouse movement corresponding to the detected combination of the hand gesture and the arm gesture comprises: When the arm motion is a movement below a predetermined threshold in the palm facing down, the hand motion is a motion to move the palm up and down while the wrist is fixed, 3 to confirm by the mouse motion to turn the wheel of the mouse down 3D mouse control method. The method of claim 12, wherein identifying a mouse movement corresponding to the detected combination of the hand gesture and the arm gesture comprises: 3D mouse control for confirming that the selected object is a mouse motion that rotates the selected object in the same rotation direction when the result of the detection of the hand motion sensor is the fist motion and the result of the detection of the arm motion sensor is the rotation of the forearm. Way.

Images