JP2010086367A - Positional information inputting device, positional information inputting method, program, information processing system, and electronic equipment - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To improve operability when performing an operation input in accordance with a gesture. <P>SOLUTION: A receiver 42 images a prescribed imaging range to generate a picked-up image, detects an operator's little finger from the generated picked-up image, calculates positional information of the detected little finger in the picked-up image, and inputs the calculated positional information into a PC 43. The PC 43 accordingly executes processing corresponding to the positional information input from the receiver 42. The invention is applicable, for example, to an information processing system comprising an input device which can recognize the gesture of one hand of the operator and perform an operation input based on the recognition result, and an information processor which performs processing corresponding to the operation input. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to a position information input device, a position information input method, a program, an information processing system, and an electronic device, and in particular, improves the operability of a position information input device that performs operation input according to an operator's gesture. The present invention relates to a position information input device, a position information input method, a program, an information processing system, and an electronic device.

  2. Description of the Related Art Conventionally, a gesture recognition technology that detects an area (one-handed area) where an operator's one hand is displayed from a captured image obtained by imaging one hand of the operator and moves a pointer on a display screen based on the detected one-handed area. (For example, refer nonpatent literature 1).

  Here, a conventional gesture recognition technique will be described. FIG. 1 shows the shape 1 of the operator's one hand.

  In the conventional gesture recognition technology, when the shape 1 of one hand as shown in FIG. 1 is detected from a captured image, the display screen is configured to track the movement of the trace point with the fingertip 2 of the index finger as a trace point (TracePoint). Move the top pointer.

  Then, when the operator moves the pointer on the display screen to an arbitrary position and clicks at the position of the pointer after the movement, the operator uses the hand that is not set as the trace point to perform the click. Press the button.

  In addition, there is an air tap operation technique that can be clicked in addition to moving the pointer on the display screen with only one hand (see Non-Patent Document 2, for example). In this air tap operation technique, as shown in FIG. 2, when an air tap for raising and lowering the index finger in the air is performed in the order of FIGS. 2A, 2B, and 2C, a click is performed on the display screen.

Thomas A. Mysliwiec. "FingerMouse: A Freehand Computer Pointing Interface," The University of Illinois, 1994. Daniel Vogel, Ravin Balakrishnan. "Distant Freehand Pointing and Clicking on Very Large, High Resolution Displays," University of Tronto.

  By the way, in the conventional gesture recognition technology, it is possible to input the position of one pointer, but it is not possible to cope with a so-called multi-touch interface that inputs the positions of two or more pointers.

  In addition, in the conventional air tap operation technology, when the air tap is performed, the fingertip of the index finger, which is a trace point, is moved by the operation of the air tap, so the position of the pointer on the display screen desired by the operator is desired. I can't click.

  In addition, it has been proposed to perform air tapping using fingers such as the thumb and middle finger instead of the index finger, but in that case as well, the fingertip of the forefinger moves due to the influence of the operation of the air tap. As a result, the position of the pointer is shifted.

  Therefore, it has also been proposed to store the past pointer positions for a predetermined time from the current time, and to perform a click at the position of the pointer immediately before the air tap is performed. Yes.

  However, when realizing such a configuration, the burden of installing software increases, so that the software to be installed is likely to be bugged, and unstable operation may be caused in operation input using gestures.

  The present invention has been made in view of such a situation, and is intended to improve the operability of an operation input by a gesture using an operator's hand.

  A first position information input device according to a first aspect of the present invention is a position information input device that inputs position information indicating a position indicated by an operator to a predetermined information processing apparatus, and images a predetermined imaging range. Imaging means for generating a captured image, detection means for detecting the operator's little finger from the generated captured image, and calculation means for calculating the position information of the detected little finger in the captured image; And a position information input device including an input means for inputting the calculated position information.

  Comparing means for comparing a three-dimensional model prepared in advance with the captured image may be further provided, and the detecting means is configured to detect the operator's little finger based on a comparison result by the comparing means. Can be.

  A marker is added to the operator's little finger, and the detection means can be made to detect the marker as the operator's little finger from the captured image. The position information indicating the position of the marker can be calculated.

  The calculation means may be configured to calculate a center of gravity of an area indicating the little finger of the operator in the captured image as the position information.

  A first position information input method according to a first aspect of the present invention is a position information input method for a position information input device that inputs position information indicating a position indicated by an operator to a predetermined information processing device. The information input device includes an imaging unit, a detection unit, a calculation unit, and an input unit. The imaging unit generates a captured image by imaging a predetermined imaging range, and the detection unit is generated. The operator's little finger is detected from the captured image, the calculation means calculates the position information of the detected little finger in the captured image, and the input means inputs the calculated position information. This is a position information input method including the step of:

  According to a first program of the first aspect of the present invention, a computer of a position information input device that inputs position information indicating a position indicated by an operator to a predetermined information processing device is captured by imaging a predetermined imaging range. Imaging means for generating an image; detection means for detecting the operator's little finger from the generated captured image; calculation means for calculating the position information of the detected little finger in the captured image; It is a program for functioning as an input means for inputting the position information.

  According to the first aspect of the present invention, a predetermined imaging range is imaged to generate a captured image, the operator's little finger is detected from the generated captured image, and the detected little finger of the detected little finger is The position information in the captured image is calculated, and the calculated position information is input.

  An information processing system according to a second aspect of the present invention executes a position information input device that inputs position information indicating a position indicated by an operator to a predetermined information processing device, and processing according to the input position information. An information processing system configured by the information processing apparatus, wherein the position information input device captures an image of a predetermined imaging range and generates a captured image, and the operation from the generated captured image. Detection means for detecting a person's little finger, calculation means for calculating the position information of the detected little finger in the captured image, and input means for inputting the calculated position information to the information processing apparatus. And an information processing system in which the information processing apparatus includes processing means for executing processing according to the input position information.

  According to the second aspect of the present invention, in the position information input device, a predetermined imaging range is imaged to generate a captured image, and the operator's little finger is detected and detected from the generated captured image. The position information of the little finger in the captured image is calculated, and the calculated position information is input to the information processing apparatus. Then, in the information processing apparatus, processing according to the input position information is executed.

  A second position information input device according to a third aspect of the present invention is a position information input device that inputs position information indicating a position indicated by an operator to a predetermined information processing apparatus, and is generated on the little finger of the operator. The position information input device includes: acceleration detection means for detecting the acceleration; update means for updating the position information based on the detected acceleration; and input means for inputting the updated position information.

  A second position information input method according to a third aspect of the present invention is a position information input method for a position information input device that inputs position information indicating a position indicated by an operator to a predetermined information processing device. The information input device includes an acceleration detection means, an update means, and an input means, wherein the acceleration detection means detects an acceleration generated in the operator's little finger, and the update means detects the detected acceleration. The position information input method includes updating the position information based on the input means, and the input unit inputting the updated position information.

  According to a second program of the third aspect of the present invention, a computer of a position information input device that inputs position information indicating a position indicated by an operator to a predetermined information processing device is transmitted to an acceleration generated on the operator's little finger. Is a program for functioning as an acceleration detecting means for detecting the position, an updating means for updating the position information based on the detected acceleration, and an input means for inputting the updated position information.

  According to the third aspect of the present invention, acceleration generated in the operator's little finger is detected, position information is updated based on the detected acceleration, and the updated position information is input.

  An electronic apparatus according to a fourth aspect of the present invention is an electronic apparatus that executes processing according to position information indicating a position designated by an operator, and an imaging unit that captures a predetermined imaging range and generates a captured image. A detection unit that detects the operator's little finger from the generated captured image; a calculation unit that calculates the position information of the detected little finger in the captured image; and And an electronic device including processing means for executing the processing.

  According to the fourth aspect of the present invention, a predetermined imaging range is imaged to generate a captured image, an operator's little finger is detected from the generated captured image, and the imaging of the detected little finger is detected. Position information in the image is calculated, and processing according to the calculated position information is executed.

  ADVANTAGE OF THE INVENTION According to this invention, the operativity of the operation input by the gesture using an operator's hand can be improved.

Hereinafter, the best mode for carrying out the invention (hereinafter referred to as an embodiment) will be described. The description will be given in the following order.

1. First Embodiment (Information Processing System: Example Using Image Processing)
2. Second Embodiment (Information Processing System: Example Using Motion Sensor)
3. Modified example of the first embodiment

<1. First Embodiment>
[External appearance of information processing system]
FIG. 3 shows an example of the appearance of the information processing system 21 according to the first embodiment of the present invention.

  In this information processing system 21, as an operator's gesture using one hand, for example, when the operator moves one hand, a pointer on the display screen moves so as to track the movement, for example. Yes.

  The information processing system 21 includes an operation input device 41, a receiving device 42 (position information input device), and a personal computer 43 (hereinafter referred to as PC 43).

  The operation input device 41 is attached to one hand of the operator, and includes a click detection unit 41a, an LED (light emitting diode) 41b, and a control unit 41c.

  The click detection unit 41a is attached to the index finger of the operator, detects that the index finger is pressed by the thumb by pressing the index finger against the thumb, and supplies the detection result to the control unit 41c.

  The LED 41b is attached to the operator's little finger (any part from the fingertip to the base of the little finger), and emits light to cause the receiving device 42 to detect the position of the operator's little finger. In the present invention, it is important to attach the LED 41b to the operator's little finger (details will be described later).

  The control unit 41c is attached to the operator's wrist, and in response to the detection result supplied from the click detection unit 41a, a click signal indicating that the click has been performed is transmitted by radio such as infrared rays. It transmits to the receiving device 42.

  For example, the receiving device 42 is fixed to the lower side of the display screen 43a of the PC 43 and is connected to the PC 43 by a wired line such as USB (universal serial bus), and receives the click signal from the control unit 41c, and receives the USB signal. Supplied to the PC 43 through the above.

  The receiving device 42 has a built-in camera that captures a predetermined imaging range, and the LED 41b (light emitted from the LED 41b) is captured by the built-in camera. Based on the captured image obtained by the imaging, coordinates (position information) indicating the position of the LED 41b in the captured image are calculated and supplied to the PC 43 via the USB or the like.

  The PC 43 is connected to the receiving device 42 by a wired connection such as USB, and moves the pointer on the display screen 43 a in accordance with the coordinates from the receiving device 42.

  In response to the click signal from the receiving device 42, the PC 43 executes a process corresponding to the click performed at the current pointer position.

[Configuration example of receiver and PC]
Next, FIG. 4 shows a detailed configuration example of the receiving device 42 and the PC 43.

  The reception device 42 includes a camera 61, a DSP (digital signal processor) 62, and a transmission unit 63.

  The camera 61 captures an image within a predetermined imaging range, and supplies a captured image obtained by the imaging to the DSP 62. Note that when the operator performs an operation of moving the pointer on the display screen 43 a or the like, it is necessary to move the one hand on which the operation input device 41 is mounted within the imaging range of the camera 61. In this case, the camera 61 obtains a captured image in which one hand of the operator is displayed by imaging and supplies the captured image to the DSP 62.

  The DSP 62 performs image processing on the captured image from the camera 61, calculates coordinates indicating the position of the operator's little finger (light emitted from the LED 41b attached to the little finger) in the captured image, and transmits the transmission unit 63. To supply.

  The transmission unit 63 transmits the coordinates from the DSP 62 to the PC 43. Note that a click signal is supplied to the transmission unit 63 from the reception device 42. The transmission unit 63 transmits the click signal from the reception device 42 to the PC 43.

  The PC 43 includes a receiving unit 81 and an application unit 82.

  The receiving unit 81 receives the coordinates and click signal from the transmitting unit 63 and supplies them to the application unit 82.

  The application unit 82 moves the pointer on the display screen 43 a of the PC 43 in accordance with the coordinates from the receiving unit 81. That is, for example, the application unit 82 detects the moving direction, moving amount, moving speed, and the like of the movement of the LED 41b based on the coordinates from the receiving unit 81, and sets the detected moving amount and moving speed in the detected moving direction. The pointer on the display screen 43a is moved by the corresponding distance.

  In response to the click signal from the receiving unit 81, the application unit 82 executes processing corresponding to the click performed at the current pointer position.

[Description of image processing performed by DSP]
Next, image processing performed by the DSP 62 will be described with reference to FIGS.

  FIG. 5 includes a position detection unit 101 and a coordinate calculation unit 102 as functional blocks realized by the DSP 62 executing a predetermined program.

  A captured image is supplied from the camera 61 to the position detection unit 101. The position detection unit 101 detects the position of the operator's little finger based on the captured image from the camera 61 and supplies the detected position to the coordinate calculation unit 102.

  Next, an example of detailed processing performed by the position detection unit 101 will be described with reference to FIG.

  FIG. 6 shows an example of a captured image captured by the camera 61. In the captured image 121 shown in FIG. 6, the LED 41b mounted on the operator's little finger is captured.

  The position detection unit 101 detects four pixels corresponding to each of the top four luminance values in the descending order of the luminance value among the luminance values of a plurality of pixels constituting the captured image 121 from the camera 61. Then, the detected positions of the four pixels are supplied to the coordinate calculation unit 102 as the position of the LED 41b (the position of the operator's little finger).

  In the case of taking the posture in which the thumb and the fingertip of the index finger are in contact with each other as in the example shown in FIG. 6, the operator touches the thumb and the fingertip of the index finger in an intuitive (sensory) manner. The point is recognized as a trace point and tends to move one hand. Therefore, it is desirable that the LED 41b mounted for detecting the trace point is mounted at a position where a trace similar to the trace of the contact point is drawn.

  Further, the LED 41b may be mounted at a position that is not affected by the click operation in order to prevent the pointer from being displaced when the click operation of pressing the click detection unit 41a is performed by pressing the index finger against the thumb. desirable.

  Therefore, the present inventor paid attention to the position of the little finger as the optimum position adopted as the trace point.

  That is, for example, when the contact point is moved as an intuitive (sensory) trace point, one hand is moved with the wrist as the rotation axis. In this case, when the distance from the wrist that is the rotation axis to the position of the little finger that is the trace point is the distance from the wrist to the contact point, the movement direction and amount of movement of the position of the little finger that is the trace point Is the same as the moving direction and moving amount of the contact point.

  The same can be said for the thumb, index finger, middle finger, and ring finger, but when the click operation is performed, the thumb, index finger, middle finger, and ring finger move against the user's intention. It is impossible to prevent the pointer from being displaced.

  Specifically, for example, when either one of the thumb or the index finger is used as a trace point, the click operation is performed using the thumb and the index finger, so that the thumb and the index finger move according to the click operation.

  Also, for example, when the click operation is performed when either the middle finger or the ring finger is used as a trace point, the middle finger and the ring finger are affected by the click operation using the thumb and the index finger due to the structure of the human hand. Since it is easy, the middle finger and the ring finger also move with the click operation.

  On the other hand, the little finger is not (almost) affected by the click operation due to the structure of the human hand.

  In addition to the little finger, the back of the hand and the palm of the hand are not affected by the click action, but the distance between the back of the hand or the palm and the wrist is shorter than the distance between the contact point and the wrist. Therefore, when the position of either the back of the hand or the palm is used as the trace point, the amount of movement of either the back of the hand or the palm that is the trace point is smaller than the amount of movement of the contact point. It is not the same as the amount of point movement.

  For this reason, as described above, the present inventor employs the position of the little finger that is moved by the same movement direction and movement amount as the contact point and is not affected by the click operation as the optimum position to be adopted as the trace point. It was decided.

  In the following description, it is assumed that the LED 41b is mounted at the position of the little finger described above as the optimum position for drawing the same locus as the locus of the contact point.

  The coordinate calculation unit 102 calculates coordinates indicating the position of the LED 41 b in the captured image 121 based on the position of the LED 41 b from the position detection unit 101, and supplies the calculated coordinate to the transmission unit 63.

  Next, an example of detailed processing performed by the coordinate calculation unit 102 will be described with reference to FIG.

  FIG. 7 shows XY coordinates defined in the captured image 121. The XY coordinates are orthogonal coordinates defined by an X axis and a Y axis that are orthogonal to each other, with the lowest left position of the captured image 121 being the origin (0, 0).

  The coordinate calculation unit 102 calculates the coordinates (x, y) on the captured image 121 corresponding to the positions of the LEDs 41b of the four pixels from the position detection unit 101 (for example, the positions 121a to 121d shown in FIG. 7). Then, for example, the center of gravity of the calculated four coordinates (x, y) is supplied to the transmission unit 63 as coordinates indicating the position of the LED 41b.

[Description of receiver operation]
Next, the first coordinate calculation process performed by the receiving device 42 will be described with reference to the flowchart of FIG.

  In step S <b> 1, the camera 61 captures an image within a predetermined imaging range, and supplies a captured image obtained by the imaging to the position detection unit 101 of the DSP 62.

  In step S <b> 2, the position detection unit 101 detects four pixels corresponding to the four highest luminance values in the descending order of the luminance value among the luminance values of the plurality of pixels constituting the captured image 121 from the camera 61. . Then, the detected positions of the four pixels are supplied to the coordinate calculation unit 102 as the position of the LED 41b (the position of the operator's little finger).

  In step S3, the coordinate calculation unit 102 calculates the coordinates (x, y) on the captured image 121 corresponding to the positions of the LEDs 41b of the four pixels from the position detection unit 101 (for example, the positions 121a to 121d shown in FIG. 7). calculate. Then, for example, the center of gravity of the calculated four coordinates (x, y) is supplied to the transmission unit 63 as coordinates indicating the position of the LED 41b.

  In step S <b> 4, the transmission unit 63 transmits the coordinates from the coordinate calculation unit 102 to the PC 43. At this time, the PC 43 detects the movement direction, movement amount, movement speed, and the like of the movement of the LED 41b based on the coordinates from the transmission unit 63, and the distance corresponding to the detected movement amount and movement speed in the detected movement direction. Only the pointer on the display screen 43a is moved.

  As described above, in the first coordinate calculation process, the position of the little finger that moves with the same movement direction and movement amount as the contact point and is not (almost) affected by the click operation is used as the trace point, and the trace point is indicated. The coordinates were calculated and transmitted to the PC 43. Therefore, the operability such as movement of the pointer and click can be further improved as compared with the case where other fingers are used as the trace points.

  That is, for example, the little finger that is the trace point does not move due to the click operation, so that it is possible to prevent the displacement of the trace point that occurs during the click operation.

  Further, when the LED 41b is attached to each of the little fingers of both hands, the two pointers can be moved at the same time and the click can be performed, so that a multi-touch interface can be supported.

  Further, the LED 41b mounted on the operator's little finger draws (almost) the same locus as the locus drawn by the contact point between the thumb and the index finger with the wrist as the rotation axis. Therefore, the operator can easily perform the operation while recognizing the contact point between the thumb and the index finger that are intuitively recognized as the trace point as the trace point.

  In addition, the operator, for example, moves one pointer to the shape of the mouse by moving the pointer with the mouse so that the one hand is in a state along the back of the one hand (from the back of the hand and the elbow). It is not necessary to maintain an unnatural state in which the wrist angle formed by the arm up to the wrist is small, and a state in which the shape of one hand is maintained that does not burden the operator (the wrist angle is large (160 to 180 degrees). It is possible to move the pointer in a natural state).

  Next, referring to FIGS. 9 to 13, the information processing system 21 using the trace point as the little finger is more operable than the case where the trace point is used as a finger other than the little finger (for example, the index finger). It is explained that the system can improve the system.

[External appearance of information processing system]
FIG. 9 shows an example of the appearance of the information processing system 141 using the trace point as an index finger.

  This information processing system 141 is the same as the information processing system 21 of FIG. 3 except that the click detection unit 41a is attached to the middle finger instead of the index finger and the LED 41b is attached to the index finger instead of the little finger. Therefore, the description of the information processing system 141 is omitted.

[Experimental results of subjects]
Next, with reference to FIG. 10 to FIG. 13, it will be described that when the little finger is used as a trace point, the accuracy of performing a click at a predetermined position is improved as compared with, for example, the case where the index finger is used as a trace point.

  FIG. 10 shows an information processing system 21 using a little finger as a trace point and an experiment in which the subject 1 is clicked alternately at two different positions for 10 reciprocations, and the index finger is used as a trace point. The experimental result when it is performed by the processing system 141 is shown.

On the display screen 43a shown in FIGS. 10A and 10B, left region T _L, and the right area T _R is displayed. The area T _L and the region T _R indicates a circle having a diameter of 7 (mm), the region T _L and the region T _R are spaced apart by 70 (mm). A black dot in the display screen 43a indicates the click position of the pointer when the operator clicks.

  The display screen 43a has a size of 11 inches and a resolution of 1280 × 768 (horizontal × vertical) pixels.

Subject 1, using an information processing system 21 and the information processing system 141, the operation of clicking alternating regions T _L and a region T _R to be displayed on the display screen 43a, carried by 10 round trip. This experiment is performed in a state where the operation input device 41 and the display screen 43a are separated by a distance of 300 (mm).

If little finger experiments were carried out by the information processing system 21, trace points (first embodiment) and, as shown in FIG. 10A, the click position of the subject 1, all in the region T _L and the region T _R include.

If the index finger experiments were carried out by the information processing system 141 to trace points, as shown in FIG. 10B, the click position of the subject 1 has gone off the area T _L and the region T _R.

  Next, FIGS. 11 to 13 show experimental results when the above-described experiment is performed on subjects 2 to 4, respectively.

  Subjects 1 to 4 represent different subjects. Further, FIGS. 11 to 13 are configured in the same manner as FIG. 10 described above, and thus description thereof is omitted.

In FIGS. 10 to 13, in any of the subject 1 to the subject 4, towards the information processing system 21, trace points little finger, it can be seen that good precision of the click position for regions T _L and the region T _R .

  In the first embodiment described above, the coordinates indicating the position of the operator's little finger are calculated by performing image processing on the captured image obtained by imaging the operator's little finger (LED 41b). As shown in FIGS. 14 to 16, it is possible to detect triaxial acceleration or the like generated in accordance with the movement of the operator's little finger, and calculate coordinates based on the detected acceleration or the like.

<2. Second Embodiment>
[External appearance of information processing system]
FIG. 14 shows an example of the appearance of an information processing system 151 according to the second embodiment of the present invention.

  In the drawing, the same reference numerals are given to the portions corresponding to the first appearance example shown in FIG. 3, and the description thereof will be omitted as appropriate. That is, the information processing system 151 according to the second embodiment is different from the information processing system 21 according to the first embodiment (FIG. 3) in that a coordinate input unit 161 (position information input device) is used instead of the LED 41b. However, instead of the receiving device 42, a receiving device 162 is provided.

  The coordinate input unit 161 is attached to the operator's little finger, and coordinates indicating the position of the operator's little finger are based on motion information indicating acceleration in three axial directions generated according to the movement of the operator's little finger. Calculate (update) and transmit to the receiving device 162.

  The receiving device 162 is fixed to the lower side of the display screen 43a of the PC 43, receives the click signal from the control unit 41c, and supplies it to the PC 43.

  The receiving device 162 receives coordinates from the coordinate input unit 161 and supplies the coordinates to the PC 43 via a wired line such as USB.

[Configuration example of coordinate input section]
FIG. 15 shows a configuration example of the coordinate input unit 161.

  The coordinate input unit 161 includes a motion sensor 181, a coordinate calculation unit 182, and a transmission unit 183.

  The motion sensor 181 is an acceleration sensor or a gyro sensor. The motion sensor 181 detects acceleration in three axial directions generated according to the movement of the operator's little finger, and supplies motion information indicating the detection result to the coordinate calculation unit 182.

  The coordinate calculation unit 182 calculates (updates) coordinates indicating the position of the operator's little finger based on the motion information from the motion sensor 181, and supplies the calculated coordinate to the transmission unit 183.

  The transmission unit 183 transmits the coordinates from the coordinate calculation unit 182 to the reception device 162.

[Explanation of operation of coordinate input unit]
Next, the second coordinate calculation process performed by the coordinate input unit 161 will be described with reference to the flowchart of FIG.

  In step S <b> 31, the motion sensor 181 is an acceleration sensor or a gyro sensor, detects acceleration in the three-axis directions according to the movement of the operator's little finger, etc., and uses the coordinate calculation unit 182 as movement information indicating the detection result. To supply.

  In step S <b> 32, the coordinate calculation unit 182 calculates (updates) coordinates indicating the position of the operator's little finger based on the motion information from the motion sensor 181, and supplies the coordinate to the transmission unit 183.

  In step S <b> 33, the transmission unit 183 transmits the coordinates from the coordinate calculation unit 182 to the reception device 162. At this time, the receiving device 162 supplies the coordinates from the transmission unit 183 to the PC 43. Then, the PC 43 moves the pointer on the display screen 43a in accordance with the coordinates from the receiving device 162.

  As described above, in the second coordinate calculation processing, the position of the little finger that is moved in the same movement direction and movement amount as the contact point and is not (almost) affected by the click operation is used as the trace point, and the trace point is indicated. The coordinates are calculated and transmitted to the PC 43 via the receiving device 162. Therefore, the operability such as movement of the pointer and click can be further improved as compared with the case where other fingers are used as the trace points.

<3. Modification of First Embodiment>

  Note that in the first embodiment, the position detection unit 101 in FIG. 5 has the luminance values up to the top four in the descending order of the luminance values among the luminance values of the plurality of pixels constituting the captured image 121 from the camera 61. The positions of 4 pixels corresponding to each were detected.

  However, if the detected luminance value of the four pixels is not within the range assumed as the luminance value of the light emitted from the LED 41b, the detected position of each of the four pixels may not be detected as the position of the LED 41b. Is possible. In this case, it is possible to prevent the position of the LED 41b from being erroneously detected.

  In addition, the number of pixels detected by the position detection unit 101 is not limited to four, and a number of pixels of 5 or more or 3 or less can be employed.

  Further, in the first embodiment, the coordinate calculation unit 102 calculates the center of gravity of the coordinates (x, y) on the captured image 121 corresponding to the position of the LED 41b of each of the four pixels from the position detection unit 101. Although calculated as coordinates indicating the position, the present invention is not limited to this.

  That is, for example, the coordinate calculation unit 102 uses the coordinates (x, y) on the captured image 121 corresponding to the position of the LED 41b of each of the four pixels as it is, the transmission unit 63 of the reception device 42, and the reception unit 81 of the PC 43. May be supplied to the application unit 82. And the application part 82 calculates the coordinate which shows the position of LED41b from the coordinate (x, y) on the captured image 121 corresponding to the position of LED41b of each four pixels from the coordinate calculation part 102, and calculated coordinates In response to this, the pointer on the display screen 43a is moved.

  In this case, the coordinate calculation unit 102 can perform processing more quickly than in the case where the center of gravity of the coordinates (x, y) on the captured image 121 is calculated.

  In the first embodiment, the LED 41b is mounted on the operator's little finger as a marker (marker) for detecting the position of the operator's little finger, and the position detection unit 101 detects the position of the LED 41b from the captured image. Is detected as the position of the operator's little finger, but the mark for detecting the operator's little finger is not limited to this.

  That is, as a mark for detecting the operator's little finger, for example, a ring fitted on the operator's little finger, a manicure reflecting a predetermined wavelength applied to the little finger, or the like can be adopted. .

  Note that when the nail polish applied to the little finger is detected as the position of the operator's little finger, the operation input device 41 becomes unnecessary, and the movement of one hand is not restricted as compared with the case where the LED 41b is attached. For example, other operations such as writing a character can be easily performed.

  Furthermore, in the first embodiment, the LED 41b is mounted on the operator's little finger in order to detect the position of the operator's little finger, but a skeleton model, a volumetric model, etc. By using this three-dimensional model, it is possible to detect the position of the joint of the operator's little finger without wearing (adding) a mark such as the LED 41b.

  That is, for example, the position detection unit 101 in FIG. 5 generates a silhouette indicating the shape of the operator's hand from a captured image obtained by imaging the operator's hand, and generates a plurality of silhouettes indicating the generated silhouette and various hand shapes. The three-dimensional model that is most similar (matched) to the generated silhouette is determined as a three-dimensional model representing the shape of the operator's hand.

  Then, by applying an extended Kalman filter (EKF) to the determined three-dimensional model, it is possible to detect the three-dimensional position of each joint of the little finger constituting the operator's hand. . In this case, as in the case where the nail polish applied to the little finger is detected as the position of the operator's little finger, the operation input device 41 is not necessary, and other operations such as writing characters can be easily performed, for example. It becomes possible.

  A more detailed detection method for detecting the three-dimensional position of a joint is, for example, N. Shimada, Y. Shirai, Y. Kuno, J. Miura, “Hand gesture estimation and model refinement using monocular camera-ambiguilty limitation by inequality constraints, “” Proc. IEEE International Conference on Automatic Face and Gesture Recognition '98, pp. 268-273, 1998 ”.

  The extended Kalman filter is described in more detail in R. Kalman, “A new approach to linear filtering and prediction problem,” Journal of Basic Engineering, Vol. 82, pp. 35-45, 1960.

  Further, the position detection unit 101 detects the position of the LED 41b as the position of the operator's little finger. For example, the center of gravity of the region indicating the operator's little finger in the captured image is determined by the operator's little finger. It can be detected as a position.

  In this case, for example, even when the operator moves the little finger without intending to move the pointer, by adopting the center of gravity as the position of the operator's little finger, It is possible to prevent the pointer from moving greatly.

  In the first embodiment, the receiving device 42 and the PC 43 are separate devices in the information processing system 21, but the receiving device 42 and the PC 43 can be configured as an electronic device having both functions. It is.

  The present invention recognizes a gesture of one hand of the operator, and based on the recognition result, in addition to a PC, a television receiver, a car navigation device, a digital photo frame that displays an image like a photo frame, etc. The present invention can be applied to an input device that can input an operation to the display control device.

  Next, the series of processes described above can be executed by dedicated hardware or can be executed by software. When a series of processing is executed by software, a program constituting the software can execute various functions by installing a so-called embedded computer or various programs. For example, it is installed from a recording medium in a general-purpose personal computer.

[Computer configuration example]
FIG. 17 shows a configuration example of a computer that executes the above-described series of processing by a program.

  A CPU (Central Processing Unit) 201 executes various processes according to a program stored in a ROM (Read Only Memory) 202 or a storage unit 208. A RAM (Random Access Memory) 203 appropriately stores programs executed by the CPU 201 and data. The CPU 201, ROM 202, and RAM 203 are connected to each other via a bus 204.

  An input / output interface 205 is also connected to the CPU 201 via the bus 204. The input / output interface 205 is connected to an input unit 206 including a keyboard, a mouse, and a microphone, and an output unit 207 including a display and a speaker. The CPU 201 executes various processes in response to commands input from the input unit 206. Then, the CPU 201 outputs the processing result to the output unit 207.

  A storage unit 208 connected to the input / output interface 205 includes, for example, a hard disk, and stores programs executed by the CPU 201 and various data. The communication unit 209 communicates with an external device via a network such as the Internet or a local area network.

  Further, a program may be acquired via the communication unit 209 and stored in the storage unit 208.

  The drive 210 connected to the input / output interface 205 drives a removable medium 211 such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory, and drives the programs and data recorded therein. Get etc. The acquired program and data are transferred to and stored in the storage unit 208 as necessary.

  As shown in FIG. 17, a recording medium for recording a program installed in a computer and executable by the computer includes a magnetic disk (including a flexible disk), an optical disk (CD-ROM (Compact Disc-Read Only Memory). ), DVD (including Digital Versatile Disc)), magneto-optical disc (including MD (Mini-Disc)), or removable media 211, which is a package media composed of semiconductor memory, or the program is temporarily or permanently It is configured by a ROM 202 to be recorded, a hard disk constituting the storage unit 208, or the like. Recording of a program on a recording medium is performed using a wired or wireless communication medium such as a local area network, the Internet, or digital satellite broadcasting via a communication unit 209 that is an interface such as a router or a modem as necessary. Is called.

  In the present specification, the step of describing the program recorded on the recording medium is not limited to the processing performed in chronological order according to the described order, but is not necessarily performed in chronological order. It also includes processes that are executed individually.

  Further, in this specification, the system represents the entire apparatus constituted by a plurality of apparatuses.

  The present embodiment is not limited to the above-described embodiment, and various modifications can be made without departing from the gist of the present invention.

It is a figure for demonstrating a gesture recognition technique. It is a figure for demonstrating an air tap operation technique. It is a figure which shows an example of the external appearance of the information processing system which is 1st Embodiment. It is a block diagram which shows the detailed structural example of a receiver and PC. It is a block diagram which shows an example of the control part implement | achieved by DSP.・ It is a figure which shows the captured image imaged with the camera. It is a figure which shows the XY coordinate defined in a captured image. It is a flowchart explaining a 1st coordinate calculation process. It is a figure which shows an example of the external appearance of the information processing system which uses a trace point as an index finger. It is a figure which shows the experimental result for the test subject. It is a figure which shows the experimental result for the test subject. It is a figure which shows the experimental result for the test subject. It is a figure which shows the experimental result for the test subject. It is a figure which shows an example of the external appearance of the information processing system which is 2nd Embodiment. It is a block diagram which shows the structural example of a coordinate input part. It is a flowchart explaining a 2nd coordinate calculation process. It is a block diagram which shows the structural example of a computer.

Explanation of symbols

  21 information processing system, 41 operation input device, 41a click detection unit, 41b LED, 41c control unit, 42 reception device, 43 PC, 61 camera, 62 DSP, 63 transmission unit, 81 reception unit, 82 application unit, 101 position detection Unit, 102 coordinate calculation unit, 151 information processing system, 161 coordinate input unit, 162 receiver, 181 motion sensor, 182 coordinate calculation unit, 183 transmission unit

Claims (11)

In a position information input device for inputting position information indicating a position indicated by an operator to a predetermined information processing device,
Imaging means for imaging a predetermined imaging range and generating a captured image;
Detection means for detecting the operator's little finger from the generated captured image;
Calculating means for calculating the position information of the detected little finger in the captured image;
A position information input device comprising: input means for inputting the calculated position information. It further includes a comparing means for comparing the prepared three-dimensional model with the captured image,
The position information input device according to claim 1, wherein the detection unit detects the operator's little finger based on a comparison result by the comparison unit. A marker is added to the little finger of the operator,
The detection means detects the marker as the operator's little finger from the captured image,
The position information input device according to claim 1, wherein the calculation unit calculates the position information indicating a position of the marker. The position information input device according to claim 1, wherein the calculating unit calculates a center of gravity of an area indicating the little finger of the operator in the captured image as the position information. In a position information input method of a position information input device for inputting position information indicating a position indicated by an operator to a predetermined information processing device,
The position information input device includes:
Imaging means;
Detection means;
A calculation means;
Including input means and
The imaging means captures a predetermined imaging range and generates a captured image;
The detection means detects the operator's little finger from the generated captured image,
The calculating means calculates the position information of the detected little finger in the captured image;
A position information input method including a step in which the input means inputs the calculated position information. A computer of a position information input device for inputting position information indicating a position indicated by an operator to a predetermined information processing device;
Imaging means for imaging a predetermined imaging range and generating a captured image;
Detection means for detecting the operator's little finger from the generated captured image;
Calculating means for calculating the position information of the detected little finger in the captured image;
A program for functioning as input means for inputting the calculated position information. An information processing system including a position information input device that inputs position information indicating a position indicated by an operator to a predetermined information processing device, and the information processing device that executes processing according to the input position information. In
The position information input device is
Imaging means for imaging a predetermined imaging range and generating a captured image;
Detection means for detecting the operator's little finger from the generated captured image;
Calculating means for calculating the position information of the detected little finger in the captured image;
Input means for inputting the calculated position information to the information processing apparatus,
The information processing apparatus is
An information processing system including processing means for executing processing according to the input position information. In a position information input device for inputting position information indicating a position indicated by an operator to a predetermined information processing device,
Acceleration detecting means for detecting acceleration generated in the operator's little finger;
Updating means for updating the position information based on the detected acceleration;
A position information input device comprising: input means for inputting the updated position information. In a position information input method of a position information input device for inputting position information indicating a position indicated by an operator to a predetermined information processing device,
The position information input device includes:
Acceleration detection means;
Update means;
Including input means and
The acceleration detecting means detects an acceleration generated in the operator's little finger;
The update means updates the position information based on the detected acceleration,
A position information input method including a step in which the input means inputs the updated position information. A computer of a position information input device for inputting position information indicating a position indicated by an operator to a predetermined information processing device;
Acceleration detecting means for detecting acceleration generated in the operator's little finger;
Updating means for updating the position information based on the detected acceleration;
A program for functioning as an input means for inputting the updated position information. In an electronic device that executes processing according to position information indicating a position indicated by an operator,
Imaging means for imaging a predetermined imaging range and generating a captured image;
Detection means for detecting the operator's little finger from the generated captured image;
Calculating means for calculating the position information of the detected little finger in the captured image;
Processing means for executing processing according to the calculated position information.

Images