JP2014052975A - Touch sense and force sense presenting device, and touch sense and force sense presenting system using the device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a touch sense and force sense presenting device capable of presenting a touch feeling and an elastic feeling, for example "a tracing movement" and "a stroking movement", or "smooth or rough", "spongy" and the like.SOLUTION: A touch sense and force sense presenting device 30 uses a tangent plane inclination obtained by a top board 38, because in squeezing, grabbing, beating, stroking movements of a person's touch sensation behavior, relative motion becomes large in a vertical direction against a finger contact surface and presentation of normal force becomes necessary to experience the touch sensation behavior. In a touch sense and force sense system, when finger tips of an operator 10 displayed in a display device 50 and photographed with a web camera 20 touch a CG object obtained by AR of an integration computer 40, the top board 38 presents the tangent plane inclination of the object toward the touching finger tips and returns impact shock from the object that is reactive force in a normal direction.

Description

The present invention relates to a force feedback device (Haptics Device) and a tactile / force sense presentation system using the same.
In particular, the present invention relates to a tactile / force sense presentation device capable of presenting a delicate and / or subtle tactile / force sense by plane approximation and a tactile / force sense presentation system using the tactile / force sense presentation system.

A force feedback device (haptic device) that presents the presence or impact of a virtual object to a person is known (for example, Japanese Patent Application Laid-Open No. 2005-190465).
Also, for example, in the fields of medical care and amusement, virtual reality (VR) or augmented reality (AR) as a variation of virtual reality is used in operations and conferences between remote locations. ) Based computer graphics (CG) are shared in the same space, touching the object of the CG, and transmitting tactile sensations, it is intuitive for the user (operator) and three-dimensionally It is known that an object can be operated (for example, Patent Document 2: Japanese Patent Laid-Open No. 11-219100).
Using such a technique, the kite can operate and discuss as if multiple people are working together in the same space.

  Also, a simple and highly realistic device is known in which visual and auditory presentations and force / tactile presentations are integrated and an actuator and a display are arranged (for example, Patent Document 3, JP 2004-54694 A).

  Furthermore, as another example of such technology, there is known an “installable force / tactile display” in which a pen or a ball, which is a contact point with a hand, is driven by a motor or a wire with a ground as a fulcrum, and a force or a tactile sensation can be presented. (For example, PHANToM, SPIDAR-G).

FIG. 1 illustrates an example of a device named SPIDE developed by Tokyo Institute of Technology as a first example of such a force / tactile device.
For example, this device displays a virtual object CG on a display, and when the user operates the grip, detects the position and posture of the grip operated by the user with a plurality of wires and a plurality of motors, and transmits them to the computer. . Information on contact and collision in the virtual space is converted into force sense by SPIDER.

FIG. 2 illustrates a device called PHANToM as a second example of such a force / tactile device.
In this apparatus, for example, when performing a simulation of surgery, when a user grips and operates a pen-type robot arm while viewing the CG displayed on the display, the CG object is transferred from the apparatus to the robot arm. The force on the hand is transmitted from the object when touched.

As described above, in FIG. 1 and FIG. 2, the force / tactile device is arranged on the desk, and the operator performs a desired operation while viewing the CG displayed on the display.
From a force / tactile device such as SPIDER or PHANToM, a force applied to the hand is transmitted from the object when the CG object is touched.
In this way, the haptic / tactile device basically transmits elasticity from a CG object when it is touched.

JP 2005-190465 A JP-A-11-219100 JP 2004-54694 A

However, the conventional haptic / tactile device cannot present even more delicate tactile sensations, such as “tracing action” or “sliding action”.
In addition, the conventional haptic / tactile device cannot present a touch such as “smooth or rough”, or a delicate touch or elasticity such as “fluffy”.

From the above, it is desired to provide a delicate tactile sensation, delicate tactile sensation, and elasticity.
In other words, CG objects based on AR are shared in the same space, and a subtle tactile sense such as “race movement” or “movement with a heel”, or a touch such as “smooth or rough”, “fluffy” By expressing the sensation and tactile sensation of actually touching a CG object more realistically while feeling a delicate touch and elasticity, such as the above, the shape of the CG object can be expressed and presented more intuitively. As a result, it is desired to present (present) a tactile sensation as if the operator is actually touching an object.

  According to the present invention, the top plate is supported on the tangential plane of the surface of the CG object in order to support the top plate and to present the normal force of the surface of the CG object that has been CG imaged. A top plate drive mechanism having first and second rotation mechanisms rotatable at least in an orthogonal direction, capable of tilting, a rotation mechanism actuator for rotating the first and second rotation mechanisms, and the actuator There is provided a tactile / force sense presentation device having a control means for driving and controlling.

In a tactile / force-sensing presentation system using a tactile / force-sensing presentation device, touched when a fingertip of an operator photographed by an imaging means displayed on a display device and a CG object by AR by a signal processing means touched The top plate presents the inclination of the tangent plane of the object to the fingertip, and returns the impact from the object, that is, the reaction force in the normal direction.
The reason for this is that when you push, grab, strike, or stroke a person's tactile perception, the relative movement in the direction perpendicular to the contact surface of the finger increases. In the present invention, the inclination of the tangential plane by the top plate is used.

When the top plate is used, the operator feels that the CG object is actually grasped or pushed back at hand, so that it becomes more intuitive and easy to operate the CG object displayed on the display device. become.
Further, when the top plate is used, it is easy to transmit a sense, and therefore, it is suitable for fine work of a CG object displayed on the display device.

  Preferably, the top plate drive mechanism includes a support portion standing on a base, a first node rotatably connected to the support portion by the first rotation mechanism, and the second And a second node rotatably connected to the first node by the rotation mechanism, and the top plate is fixed to the second node.

  Preferably, the base is provided with at least one wheel that can move in two directions, each of which can move the base in a two-dimensional direction, and has a wheel actuator that drives the wheel.

  The surface of the top plate has a flat surface, is fluffy, or is formed so as to give a tactile sensation of being rough or crisp.

  According to the invention, the tactile / force-sensing presentation device, the imaging unit for imaging the tactile / force-sensing presentation device, the imaging result of the imaging unit are input, and the CG image by the AR is generated to display the display device. The top plate can be tilted to the tangential plane of the surface of the CG object in order to enable the display of the normal force of the surface of the CG object imaged in the CG image in the top plate drive mechanism. A tactile / force sensation comprising: a signal processing means for generating a control signal for controlling the rotation mechanism actuator for rotating the first and second rotation mechanisms rotatable at least in the orthogonal direction and outputting the control signal to the control means. A presentation system is provided.

  Preferably, the signal processing means generates a control signal for driving the wheel when the wheel that is movable in the two directions is provided on the base, and outputs the control signal to the control means. The wheel actuator is driven and controlled.

According to the present invention, it is possible to present a delicate tactile sensation such as, for example, “race movement” or “brow movement”.
In addition, according to the present invention, the surface of the top board can be brought into a state in which a touch such as “smooth or rough”, a delicate touch such as “fluffy”, and a feeling of elasticity are presented.

  That is, according to the present invention, CG objects based on AR are shared as if in the same space, and a subtle tactile sensation such as “tracing motion” or “sliding motion” or “smooth or rough” Expressing the shape of a CG object more intuitively by expressing the touch and sensation of touching a CG object more realistically while feeling a delicate touch such as “fluffy” and elasticity. As a result, it is possible to present (present) a tactile sensation as if the operator is actually touching the object intuitively.

FIG. 1 is a diagram illustrating an example of SPIDER as a first example of a conventional force / tactile device. FIG. 2 is a diagram illustrating PHANToM as a second example of a conventional force-tactile device. FIG. 3 is a diagram showing a schematic configuration of the haptic / tactile sensation system according to the first embodiment of the present invention. 4 is a perspective view of a tactile / force sense presentation device used in the tactile / force sense presentation system illustrated in FIG. FIG. 5 is a diagram showing an example in which the haptic / force sense presentation system illustrated in FIG. 3 is displayed on the display unit as an exemplary CG. FIG. 6 is a side view of the tactile / force sense presentation device when the top plate is operated. FIG. 7 is a side view of the movement of the top plate drive mechanism of the tactile / force sense presentation device when the top plate illustrated in FIG. 6 is operated. FIG. 8 is a perspective view when two axes of the tactile / force-sensing presentation device illustrated in FIG. 4 are rotated. FIG. 9 is a diagram showing a control system for operating the two axes of the drive mechanism of the tactile / force sense presentation device. FIG. 10 is a diagram illustrating the use of a top plate according to the application. FIG. 11 is a diagram showing a case where a color glove is used.

Hereinafter, a tactile and force sense presentation device of the present invention and an embodiment using the device will be described.
First Embodiment A first embodiment of a haptic / force-sensing presentation device (device) and a haptic / force-sensing presentation apparatus (system) using the same will be described with reference to FIGS.
FIG. 3 is a diagram showing a schematic configuration of the haptic / tactile sensation system according to the first embodiment of the present invention.
4 is a perspective view of a tactile / force sense presentation device used in the tactile / force sense presentation system illustrated in FIG.
FIG. 5 is a diagram showing an example of display on the display device as an exemplary CG in the tactile / force sense presentation system illustrated in FIG. 3.
FIG. 6 is a side view when the top plate of the tactile / force-sensing device is operated.
FIG. 7 is a side view of the movement of the mechanism of the tactile / force-sensing device when the top plate illustrated in FIG. 6 is operated.
FIG. 8 is a perspective view when two axes of the drive mechanism of the tactile / force sense presentation device illustrated in FIG. 4 are simultaneously rotated.
FIG. 9 is a diagram showing a control system including a servo controller 60 that operates the two axes of the drive mechanism of the tactile / force sense presentation device.

Configuration of Tactile / Force Display System In FIG. 3, the tactile / force display system 1 includes a web camera 20, a tactile / force display device 30, an overall computer 40, a display device 50, and a servo controller 60. Have
In this specification, the whole including the haptic / force-sensing presentation device 30 described in detail with reference to FIG. 4 is referred to as a haptic / force-sensing presentation system.
The web camera 20 corresponds to the imaging means of the present invention. The tactile / force sense presentation device 30 corresponds to the tactile / force sense presentation device of the present invention. The general computer 40 corresponds to the signal processing means of the present invention. The display device 50 corresponds to the display means of the present invention. The servo controller 60 corresponds to the control means of the present invention.

Web Camera The web camera 20 is arranged so as to be able to capture an image of the entire tactile sense / force sense presentation system.
In particular, the web camera 20 images the entire tactile / force-sensing presentation device illustrated in FIG. 4 and the first to fourth position reference markers 33-1 to 33-4 disposed on the base 31. It is arranged to be possible.

4. Configuration of Tactile / Force Display Device The tactile / force display device 30 illustrated in FIG. 4 is placed, for example, on a flat desk so as to be movable in a two-dimensional manner. Operate.
The tactile / force sense presentation device 30 includes a base 31, first to fourth omnibus wheels 32-1 to 32-4, first to fourth reference position markers 33-1 to 33-4, a ceiling It has a plate drive mechanism 34, first and second rotating shafts 36-1, 36-2, and a top plate 38.

  In addition, the 1st-4th omnibus wheel 32-1-32-4 is not a component essential to this invention.

The top board tactile / force sense presentation device 30 is provided with a flat top board 38 that the operator 10 touches.
The top plate 38 is supported by the top plate drive mechanism 34 and can be tilted freely.

Hereinafter, the reason for using the top plate 38 and the top plate drive mechanism 34 will be described.
As illustrated in FIG. 5, the fingertip of the operator 10 photographed by the web camera 20 displayed on the display device 50 and an object (CG object) displayed as a CG image by AR by the central computer 40 are touched. Then, the top plate 38 presents the inclination of the tangential plane of the surface of the CG object to the touched fingertip, and returns the impact from the CG object, that is, the reaction force in the normal direction of the surface of the CG object. The reason for this is that when you push, grab, strike, or stroke a person's tactile perception, the relative movement in the direction perpendicular to the contact surface of the finger increases. Therefore, the inclination of the tangent plane by the top plate 38 was used.
The tangent plane is defined as a smooth curved surface S and a point on S. At this time, if a curve on S passing through P is arbitrarily taken and a tangent of this curve in P is made, they are all on the same plane. This plane is called a tangent plane of the curved surface S at the point P.

When the top plate 38 is used, the operator 10 is more intuitive and operates the CG object displayed on the display device 50 because a sense of actually grasping or pushing back the CG object is generated at hand. It becomes easy.
Further, when the top plate 38 is used, the sense is easily transmitted, and therefore, it is suitable for fine work of the CG object displayed on the display device 50.

The reference position markers first to fourth reference position markers 33-1 to 33-4 are marks for detecting the position of the base 31, and are arranged at the four corners of the base 31.

Web Camera The web camera 20 includes first to fourth position reference markers 33-1 to 33-4 of the base 31, and images the entire tactile / force sense presentation device 30.
Images are captured by the web camera 20, and the imaging results are output from the web camera 20 to the central computer 40. The central computer 40 detects the positions of the four corners of the base 31 from the imaging results.
Further, the result of imaging by the web camera 20 is displayed as CG on the display device 50 via the overall computer 40 as illustrated in FIG.
The display device 50 is, for example, a liquid crystal display device.

The top plate drive mechanism The top plate 38 is supported by the top plate drive mechanism 34 so as to be freely tiltable, for example, as illustrated in FIG.
The top plate drive mechanism 34 is erected from the base 31 and, as illustrated in FIG. 7, includes a first pillar 34-1, a second pillar 34-2, and a third pillar 34-3. Has been.
The first column 34-1 corresponds to the support unit standing on the base of the present invention, and the second column 34-2 is rotatable with respect to the support unit by the first rotation mechanism of the present invention. Corresponding to the connected first node, the third column 34-3 corresponds to the second node rotatably connected to the first node by the second rotating mechanism of the present invention. .
The top plate 38 is fixed to the second node (third column 34-3).

The first and second rotating shafts 36-1 and 36-2 are rotatable in the orthogonal direction along the vertical axis of the top plate driving mechanism 34.
The first column 34-1 and the second column 34-2 are rotatably connected by a first rotating shaft 36-1. Similarly, the 2nd pillar 34-2 and the 3rd pillar 34-3 are connected rotatably by the 2nd axis of rotation 36-2.
The first and second rotating shafts 36-1 and 36-2 are driven by a servo controller 60 and servomotors 70-1 and 70-2 illustrated in FIG.

The example illustrated in FIGS. 6 and 7 shows an example in which the third column 34-3 is rotated by the rotation operation of the second rotation shaft 36-2, and as a result, the top plate 38 is inclined to the right side. Yes.
On the other hand, FIG. 8 shows a case where the first and second rotating shafts 36-1 and 36-2 are driven simultaneously. By driving the first and second rotating shafts 36-1 and 36-2 simultaneously and appropriately, the top plate 38 can be tilted to a desired inclination.

  With the above configuration, the top plate 38 touches the fingertip of the operator 10 photographed by the web camera 20 displayed on the display device 50 and the CG object by AR by the central computer 40 as illustrated in FIG. The top plate 38 driven by the servo controller 60 and the servo motor shown in FIG. 9 presents the inclination of the tangent plane of the object to the touched fingertip, and the impact from the object, that is, the normal line Returns the reaction force in the direction.

  As described above, when the top plate 38 is used, the operator 10 has a feeling of actually grasping or pushing back the CG object, so that the operator 10 becomes more intuitive and the CG object displayed on the display device 50 is displayed. It becomes easy to operate. Further, when the top plate 38 is used, the sense is easily transmitted, and therefore, it is suitable for fine work of the CG object displayed on the display device 50.

In the haptic / tactile sensation presentation method described above, the overall computer 40 refers to the imaging result of the web camera 20 to determine the presentation content of the tilt of the top board 38, and further, the first and second rotation axes 36. −1, 36-2 are determined and the servo controller 60 is instructed.
The servo controller 60 drives and controls the above-described top plate 38 by drivingly controlling the first servo motor 70-1 and / or the second servo motor 70-2 in accordance with a command from the general computer 40. Realization of haptic and tactile sensation by.

  The general computer 40 further drives the first to fourth omnibus wheels 32-1 to 32-4 via the servo controller 60, thereby providing the force / tactile sensation to the top board 38 described above. Thus, it is possible to add force / tactile senses in a two-dimensional plane direction.

Movement of base, omnibus wheel Preferably, the 1st-4th omnibus wheel 32-1-32-4 is arrange | positioned in the center part of each side of the base 31, respectively.
The first to fourth omnibus wheels 32-1 to 32-4 are wheels that move the base 31 on a desk in a free direction on a plane (two dimensions).
For example, the operation of the first omnibus wheel 32-1 will be described as an example. In FIG. 4, when the entire omnibus wheels 32-1 to 32-4 rotate in the first rotation direction R, the base 31 moves in the Y direction. The first omnibus wheel 32-1 is provided with a plurality of second wheels (not shown) that rotate in the direction r orthogonal to the first rotation direction R along the first rotation direction R. When the second wheels are rotated, that is, when the second wheels are rotated in the second rotation direction r orthogonal to the first rotation direction, the base 31 moves in the X direction.
When the omnibus wheel 32-1 is rotated in the first rotation direction R and at the same time the second wheel is rotated in the second rotation direction r, the base 31 moves in a direction and speed along the vector sum of both. To do.

  As described above, when the first to fourth omnibus wheels 32-1 to 32-4 are appropriately operated, the base 31 can freely move on the two-dimensional plane.

In order to enable the above-described operation, each of the omnibus wheels 32-1 to 32-4 is, for example, as illustrated in FIG. 9, the first motor M for rotating in the first rotation direction, and the second Are driven by a plurality of second motors m1 to mn for rotating in the rotation direction r.
The first motor M of each omnibus wheel 32 and the plurality of second motors m <b> 1 to mn are driven and controlled by a servo controller 60.
The servo controller 60 controls the first motor M and the plurality of second motors m <b> 1 to mn for each omnibus wheel 32 under the instruction of the overall computer 40.

  As described above, in the drive control of the first to fourth omnibus wheels 32-1 to 32-4, as in the drive control of the first and second rotary shafts 36-1 and 36-2, the overall computer 40 is connected to the web. Referring to the imaging result of the camera 20, in addition to the presentation contents of the tilt of the top board 38, the movement (direction, amount) of the base 31 is determined, and further, the first motor of each omnibus wheel, a plurality of The drive amount of each second motor is determined and the servo controller 60 is instructed. The servo controller 60 is realized by driving and controlling the servo motors 71-0 and 71-1 to 71-n in accordance with commands from the overall computer 40.

  When the first to fourth omnibus wheels 32-1 to 32-4 are operated, the top plate 38 can be moved in the plane direction, and an effect of being traced with the finger of the operator 10 can be added.

Second Embodiment FIG. 10 is a diagram showing the use of a top plate according to the application.
In the first embodiment, the case where the top plate 38 has a flat surface has been described.
When the operator 10 touches the surface of the top plate 38 according to the use and purpose, for example, the surface illustrated in FIG. 10A is “fluffy” like fur or velvet. The surface illustrated in FIG. 10 is “gritty” like sandpaper, and the surface illustrated in FIG.
When the top plate 38 having such a surface is used, when the operator 10 traces the CG object, the tactile sensation such as rough, fluffy, tingling, etc., when the CG object is stroked or traced. Can be reproduced. Therefore, the operator 10 can perceive a more accurate tactile sensation.

Third Embodiment FIG. 11 is a diagram showing a case where a color glove is used.
In the present embodiment, the position of the finger of the operator 10 is acquired using a color glove. In addition, the operation can be applied to two tactile / force sense presentation devices by two hands and five fingers.

Application Examples The tactile / force sense presentation device and the tactile / force sense presentation system using the tactile / force sense presentation device according to the embodiment of the present invention can be applied to various fields.

First application example For example, in museums, there are many exhibits that should not be touched. In such a case, when the tactile / force sense presentation device and the tactile / force sense presentation system using the tactile / force sense presentation system according to the present invention are applied, the CG object of the exhibit displayed by the AR displayed on the display device 50 is touched. , You can experience as if you are touching the actual exhibits.

Second Application Example When a CG object displayed by AR is operated using a tactile / force-sensing presentation device and a tactile / force-sensing presentation system according to the present invention in a conference in a remote place, the tactile sensation of the object is the top plate 38. CG object can be manipulated more intuitively.

Third application example Even in the learning and hobbies of art using clay, paper, and bond, the haptic / force-sensing presentation device and the corpse 1 of the present invention are applied to operate a CG object using AR to transmit the senses. By performing the operation, it is possible to create a work of art from the CG with the feeling that the operator 10 is actually touching the object.
In this case, even if the creation fails, there is no need to consider disposal because no failed works remain in the real space (the real world).

  As described above, there are various other application examples of the haptic / force-sensing presentation device of the present invention and the haptic / force-sensing presentation system using the device.

DESCRIPTION OF SYMBOLS 1 ... Tactile sense / force sense presentation system 10 ... Operator, 20 ... Web camera, 30 ... Tactile sense / force sense presentation device, 40 ... Overall computer, 50 ... Display device, 60 ... Servo controller 31 ... Base, 32-1 to 32-4 ... First to fourth omnibus wheels, 33-1 to 33-4 ... First to fourth position references Marker, 34 ... top plate drive mechanism, 38 ... top plate.

Claims (9)

With the top plate,
The top plate can be tilted to the tangential plane of the surface of the CG object so as to support the top plate and be able to present the normal force of the surface of the CG object imaged by CG. A top plate drive mechanism having first and second rotation mechanisms rotatable at least in the orthogonal direction;
A rotation mechanism actuator that rotationally drives the first and second rotation mechanisms;
Control means for driving and controlling the actuator;
Having
Tactile / force-sensing device. The top plate drive mechanism is
A support portion erected on the base;
A first node rotatably connected to the support portion by the first rotation mechanism;
A second node rotatably connected to the first node by the second rotation mechanism;
The top plate is fixed to the second node;
The tactile / force sense presentation device according to claim 1. The base is provided with at least one two-way movable wheel, each enabling the base to move in a two-dimensional direction,
A wheel actuator for driving the wheel;
The tactile / force-sensing device according to claim 1. The top plate has a flat surface.
The tactile / force sense presentation device according to claim 1. The surface of the top plate is formed to give a fluffy touch,
The tactile / force sense presentation device according to claim 1. The surface of the top plate is formed to give a rough tactile sensation,
The tactile / force sense presentation device according to claim 1. The surface of the top plate is formed to give a tactile sensation,
The tactile / force sense presentation device according to claim 1. Tactile / force-sensing presentation device according to any one of claims 1 to 7,
Imaging means for imaging the haptic / force-sensing device;
The imaging result of the imaging means is input, a CG image by AR is generated and displayed on the display device, and the normal force of the surface of the CG object that has been converted to a CG image can be presented in the top plate driving mechanism. Therefore, a control signal for controlling the rotation mechanism actuator is generated, which allows the top plate to be inclined to the tangential plane of the surface of the CG object, which rotates the first and second rotation mechanisms which can rotate at least in the orthogonal direction. And signal processing means for outputting to the control means;
A tactile and haptic presentation system. The signal processing means generates a control signal for driving the wheel when the base is provided with a wheel capable of moving in the two directions, and outputs the control signal to the control means.
The control means drives and controls the wheel actuator;
The tactile / force sense presentation system according to claim 8.

Images