JP7264844B2 - Haptic presentation system, controller and program - Google Patents

Description

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a haptic presentation system, a control device, and a program for presenting a haptic sensation that indicates the sensation of touching an object.

2. Description of the Related Art Conventionally, there is known a device for giving a product a tactile sensation (see, for example, Patent Literature 1). In the device described in Patent Document 1, the image of the product selected by the user is displayed, and by changing the amount of displacement of the touch panel depending on the product, the user can check the tactile sensation of the product.

JP 2019-128738 A

In conventional devices, an image of a product is displayed while the user touches a touch panel for presenting the user with the tactile sensation of the product. However, since only an image showing the shape of the product in its normal state is displayed, there is a problem that the user cannot grasp the shape of the product when the product is touched.

Therefore, the present invention has been made in view of these points, and it is an object of the present invention to enable the user to grasp the shape of an object when presenting a force sensation when the user touches the object.

A haptic presentation system according to a first aspect of the present invention includes a haptic presentation device that presents a haptic sensation, and a control device that controls the haptic presentation device. The haptic presentation device includes a movable part that generates a movable part reaction force having a magnitude corresponding to an amount displaced by being pushed, and a movable part that is an amount displaced by the movable part when the movable part is pushed. and a displacement amount detection unit that detects the displacement amount. The control device includes an information acquisition unit that acquires reaction force information indicating a relationship between the movable portion displacement amount and the movable portion reaction force generated in the movable portion in association with a plurality of objects; a selected object identification unit that identifies one selected object; a haptic control unit that controls a movable part; an image data acquisition unit that acquires object image data as image data of the selected object; and a deformed image by deforming the object image data based on the displacement amount of the movable part. An image data creation unit for creating data, and a display control unit for displaying the deformed image data on a display device.

The image data acquisition unit acquires a plurality of deformed image data corresponding to each of the plurality of movable portion displacement amounts, and the image data creation unit corresponds to the movable portion displacement amount from the plurality of deformed image data. The modified image data may be selected.

The display control section may cause the display device to sequentially display the deformed image data having different shapes in accordance with a change in the displacement amount of the movable portion.

The information acquisition unit may acquire the movable part displacement amount and the reaction force information indicating the elasticity and viscosity of the selected object in association with a plurality of objects.

The haptic presentation device further includes an orientation specifying unit that specifies an orientation of the haptic presentation device, and the information acquiring unit associates the respective positions of the plurality of objects with the displacement amount of the movable portion and the Acquiring the reaction force information indicating the relationship with the movable portion reaction force generated in the movable portion, the force sense control portion determines the position corresponding to the orientation identified by the orientation identification portion in the reaction force information, the The movable portion may be controlled to generate the movable portion reaction force corresponding to the amount of displacement of the movable portion.

A control device according to a second aspect of the present invention is a control device for controlling a movable portion of a force sense presentation device that presents a force sense, wherein the movable portion is associated with a plurality of objects, and when the movable portion is pushed, the movable portion is pushed. an information acquiring unit that acquires reaction force information indicating a relationship between a movable portion displacement amount that is an amount of displacement of a portion and a movable portion reaction force that is a reaction force generated in the movable portion; and a selected object identifying unit that identifies one selected object, and controls the movable part so as to generate the movable part reaction force associated with the movable part displacement amount acquired from the force sense presentation device in the reaction force information. an image data acquisition unit that acquires object image data that is image data of the selected object; and a deformed image data is created by deforming the object image data based on the movable portion displacement amount. It has an image data creation unit and a display control unit that causes a display device to display the deformed image data.

A program according to a third aspect of the present invention associates a processor of a control device that controls a movable portion of a haptic presentation device that presents a haptic with a plurality of objects, and causes the movable portion to move when the movable portion is pressed. an information acquiring unit that acquires reaction force information indicating a relationship between a movable portion displacement amount that is an amount of displacement of a portion and a movable portion reaction force that is a reaction force generated in the movable portion; A selected object identification unit that identifies one selected object, and a force that controls the movable portion so as to generate the movable portion reaction force associated with the movable portion displacement amount acquired from the force sense presentation device in the reaction force information. a sense control unit, an image data acquisition unit that acquires object image data that is image data of the selected object, and an image data creation unit that creates deformed image data by deforming the object image data based on the movable portion displacement amount. , and a display control unit for displaying the deformed image data on a display device.

ADVANTAGE OF THE INVENTION According to this invention, when presenting a force sense when a user touches an object, it is effective in the ability for a user to grasp the shape of an object.

1 is a diagram showing an outline of a haptic presentation system S; FIG. 2 is a diagram showing configurations of a haptic device 1 and a control device 2. FIG. 4A and 4B are diagrams for explaining the operation of a displacement amount detection unit 12; FIG. 4 is a diagram showing reaction force information data stored in a storage unit 22; FIG. It is a figure which shows an example of object image data. 4 is a flow chart showing the flow of processing for the control device 2 to cause the haptic presentation device 1 to present a haptic. FIG. 11 is a diagram showing an outline of a haptic presentation system S according to a modification;

[Overview of haptic presentation system S]
FIG. 1 is a diagram showing an outline of a haptic presentation system S. As shown in FIG. A haptic presentation system S includes a haptic presentation device 1 , a control device 2 , and a display device 3 . The haptic presentation system S is a system for presenting the user U with a haptic sensation, which is the feel of pressing an object that the user U does not have at hand.

The force sense presentation device 1 is a device for presenting the user U with a sense of force. The force sense presentation device 1 has a main body portion 10 and a movable portion 11 . The body portion 10 is a base for placing the force sense presentation device 1 on a desk, a floor, or the like, and has, for example, a rectangular parallelepiped shape.

The movable portion 11 is attached to the body portion 10 on one surface of the body portion 10 so as to be movable in the longitudinal direction. The movable part 11 has an actuator that generates a movable part reaction force having a magnitude corresponding to the amount of displacement caused by being pushed by the user U. As shown in FIG. For example, the movable part 11 generates a larger reaction force as the user U presses more. The movable part 11 can change the magnitude of the movable part reaction force under the control of the control device 2 .

The control device 2 controls the movable portion 11 and causes the display device 3 to display an image. The control device 2 controls the movable part 11 so as to generate a movable part reaction force corresponding to the object selected by the user U, for example. Further, the control device 2 causes the display device 3 to display the image of the object selected by the user U. FIG. At this time, the control device 2 deforms the image of the object displayed on the display device 3 as the user U presses the movable part 11 .

Specifically, the control device 2 acquires information indicating the amount of displacement of the movable portion 11 from the force sense presentation device 1, and deforms the image of the object based on the acquired amount of displacement. By operating the haptic device 1 in this way, the user U can visually recognize how the object is deformed while pressing the movable part 11 . As a result, the user U can feel as if he is actually pushing the object.

The control device 2 displays, for example, information for identifying a plurality of products that the user U can purchase before displaying the image of the object. When the user U selects one product from a plurality of products, the control device 2 causes the display device 3 to display an image of the selected product. Further, the movable part 11 is controlled so that the movable part 11 generates a movable part reaction force corresponding to the viscoelasticity of the selected product.
The configurations and operations of the force sense presentation device 1 and the control device 2 will be described in detail below.

[Configuration of haptic presentation device 1 and control device 2]
FIG. 2 is a diagram showing the configurations of the force sense presentation device 1 and the control device 2. As shown in FIG. The haptic presentation device 1 and the control device 2 are connected by a communication line such as a serial communication cable or a wireless communication line such as Bluetooth (registered trademark).

The force sense presentation device 1 has a displacement amount detection section 12 in addition to the main body section 10 and the movable section 11 described above. The displacement amount detection unit 12 detects a movable portion displacement amount, which is an amount by which the movable portion 11 is displaced when the movable portion 11 is pushed. The displacement amount detection unit 12 transmits the detected movable portion displacement amount to the control device 2 via a communication line.

3A and 3B are diagrams for explaining the operation of the displacement amount detection unit 12. FIG. 3(a) shows a state in which the movable portion 11 is not pushed, and FIG. 3(b) shows a state in which the movable portion 11 is pushed. The displacement amount detection unit 12 detects the amount of displacement in the longitudinal direction of the movable unit 11 with reference to the position in the state shown in FIG. 3(a). In the state shown in FIG. 3A, the position of the upper surface of the movable part 11 is x=x0 (m), and the movable part 11 applies a force of F=F0 (N) for compensating for gravity and frictional force. emit.

In the state shown in FIG. 3B, the position of the upper surface of the movable portion 11 is x=x1(m). In this case, the displacement amount detection unit 12 detects the displacement amount of the movable portion in the state of FIG. 3(b) as Δx=x1−x0. In this state, the movable part 11 generates a force of F=F0+F1(N), and the user feels a reaction force of F1(N).

The displacement amount detection section 12 may further detect the displacement speed of the movable section 11 . Since the movable portion 11 is not moving in the state of FIG. 3A, the displacement velocity v=0 (m/s). On the other hand, in the state of FIG. 3(b), the movable part 11 is moving, and v=v1 (m/s). The displacement amount detection unit 12 notifies the control device 2 of the detected movable portion displacement amount and displacement speed.

Movable part 11 receives reaction force information used to generate a movable part reaction force from control device 2 via a communication line with force sense presentation device 1 . The reaction force information is information indicating the relationship between the movable portion displacement amount and the movable portion reaction force. The movable part 11 generates a movable part reaction force associated with the movable part displacement amount notified from the displacement amount detection part 12 in the received reaction force information.

Next, the configuration and operation of the control device 2 will be described. The control device 2 has a communication section 21 , a storage section 22 and a control section 23 .

The communication unit 21 is a communication interface for transmitting and receiving data to and from an external device. The communication unit 21 has a communication controller for connecting to the Internet or an intranet, for example.

The storage unit 22 has a storage medium such as a ROM (Read Only Memory), a RAM (Random Access Memory), and a hard disk. The storage unit 22 stores programs executed by the control unit 23 . The storage unit 22 also stores reaction force information data including reaction force information associated with an object.

FIG. 4 is a diagram showing reaction force information data stored in the storage unit 22. As shown in FIG. The reaction force information data shown in FIG. 4 includes reaction force information corresponding to a plurality of objects. In the reaction force information data shown in FIG. 4, an information ID, an object name, an elastic coefficient, a viscosity coefficient, and an object image data name are associated. The object image data is image data of an object displayed on the display device 3 .

FIG. 5 is a diagram showing an example of object image data. In the object image data, the movable portion displacement amount is associated with the deformed image data indicating the deformed state of the object. A displacement amount of 0 is associated with deformed image data in which the object is not deformed. Deformed image data in which the object is slightly depressed is associated with the displacement amount xa. The displacement amount xb is associated with deformed image data in which the object is further depressed. These deformed image data also show an image of a finger whose tip is in contact with a concave portion of the object.

The control unit 23 has, for example, a CPU (Central Processing Unit). By executing the programs stored in the storage unit 22, the control unit 23 controls the information acquisition unit 231, the selected object identification unit 232, the haptic control unit 233, the image data acquisition unit 234, the image data generation unit 235, and the display control unit. It functions as part 236 .

The information acquisition unit 231 acquires reaction force information indicating the relationship between the movable portion displacement amount and the movable portion reaction force generated in the movable portion 11 in association with a plurality of objects. Specifically, the information acquisition unit 231 acquires reaction force information indicating the relationship between the displacement amount of the movable portion and the elasticity and viscosity of the selected object specified by the selected object specifying unit 232, which will be described later, in association with a plurality of objects. do. The information acquisition unit 231 may acquire reaction force information from an external device, or may acquire reaction force information stored in the storage unit 22 in advance, for example. The information acquisition unit 231 may simultaneously acquire a plurality of pieces of reaction force information corresponding to a plurality of objects from an external device and store them in the storage unit 22, and may acquire one piece of reaction force information corresponding to an object designated by the user U. may be obtained, and the force sense control unit 233 may be notified of the obtained reaction force information.

The selected object identification unit 232 identifies one selected object selected from a plurality of objects. The selected object specifying unit 232 acquires, for example, through the communication unit 21, information for specifying an object selected by an information terminal (for example, a smartphone or a computer) used by the user U, and based on the acquired information , to identify a selected object for which a haptic sensation is presented to the user U. The selected object specifying unit 232 notifies the haptic control unit 233 of information for specifying the specified selected object (for example, object identification information).

The haptic control section 233 controls the movable section 11 so as to generate a movable section reaction force associated with the movable section displacement amount in the reaction force information. The haptic control unit 233 determines the elastic coefficient k (N/m) and the viscosity coefficient d (Ns/m) corresponding to the selected object notified from the selected object identification unit 232 in the reaction force information data shown in FIG. The magnitude of the reaction force F when the movable portion displacement amount of the movable portion 11 is x and the speed is v is calculated using the following formula.
F = F0 + kx + dv

Here, F0 is a value determined by identifying disturbance forces such as gravity and frictional force applied to the movable portion 11, as described above. The values of k and d are also values determined by performing tests in advance and identifying the feeling when a person presses an actual object and the feeling when the moving part 11 is pressed.

The image data acquisition unit 234 acquires object image data, which is image data of the selected object. The image data acquisition unit 234 acquires a plurality of deformed image data corresponding to, for example, a plurality of movable portion displacement amounts. The image data acquisition unit 234 may acquire object image data corresponding to the selected object from an external device that stores object image data, and select from a plurality of object image data stored in the storage unit 22 in advance. Object image data corresponding to the object may be obtained.

The image data creation unit 235 creates deformed image data by deforming the object image data based on the displacement amount of the movable part. The image data creating unit 235 creates deformed image data by referring to the object image data shown in FIG. 5, for example, and selecting deformed image data corresponding to the displacement amount of the movable part from a plurality of deformed image data. The image data creating unit 235 may create deformed image data according to the displacement amount of the movable part by performing arithmetic processing on the basis of the deformed image data when the displacement amount of the movable part is 0. The image data creation unit 235 notifies the display control unit 236 of the created deformed image data.

The display control unit 236 causes the display device 3 to display the deformed image data notified from the image data creation unit 235 . The display control unit 236 causes the display device 3 to sequentially display deformed image data having different shapes in accordance with a change in the amount of displacement of the movable part. The display control unit 236 causes the display device 3 to sequentially display deformed image data corresponding to the displacement amount of the movable portion at predetermined time intervals (for example, 100 milliseconds). It is possible to allow the user U to visually recognize a moving image in which the shape of the object changes accordingly.

[Processing Flow of Control Device 2]
FIG. 6 is a flow chart showing the flow of processing in which the control device 2 causes the haptic presentation device 1 to present a haptic. The flowchart shown in FIG. 6 starts when the user U activates application software that allows the user to select an object whose touch is to be confirmed.

The information obtaining unit 231 identifies the selected object by obtaining information for identifying the object selected by the user U (S11). Subsequently, the haptic control unit 233 determines whether or not the user U has performed an operation to start presenting the haptic (S12). When the haptic control unit 233 determines that an operation to start presenting the haptic has been received (YES in S12), the haptic control unit 233 acquires initial position information indicating the initial position of the movable unit 11 from the haptic presentation device 1 (S13). .

Subsequently, the haptic controller 233 starts acquiring the displacement amount of the movable part from the haptic device 1 (S14). The force sense control unit 233 calculates the movable part reaction force corresponding to the obtained movable part displacement amount (S15), and notifies the movable part 11 of the calculated movable part reaction force, thereby determining the reaction force generated by the movable part 11. is controlled (S16).

The haptic control unit 233 also notifies the image data creation unit 235 of the displacement amount of the movable part, and the image data creation unit 235 creates deformed image data corresponding to the displacement amount of the movable part (S17). The display control unit 236 causes the display device 3 to display the deformed image data created by the image data creation unit 235 (S18).

The haptic control unit 233 confirms whether or not an operation to terminate the presentation of the haptic has been received from the user U (S19), and if the termination operation has not been received (NO in S19), the processes from S14 to S19 are performed. repeat. When the haptic control unit 233 receives an end operation, it ends the process of presenting the haptic.

By operating the force sense presentation system S as described above, the user U can confirm the feel of the object by pressing the movable portion 11 and can visually recognize how the object is deformed. Therefore, the user U can grasp the shape of the object when presenting the force sense when the user U touches the object.

[First modification]
FIG. 7 is a diagram showing an overview of a haptic presentation system S according to a modification. In the above description, the moving direction of the movable part 11 is the vertical direction, and the case where the user U pushes the movable part 11 from above to below was exemplified. The difference is that the user U pushes the movable part 11 in another direction.

The haptic presentation device 1 according to this modified example further includes an orientation specifying unit that specifies the orientation of the haptic presentation device 1 . The orientation specifying unit has, for example, a gyro sensor, and specifies the movable direction of the movable unit 11 . The orientation identification unit notifies the haptic control unit 233 and the image data generation unit 235 of the identified movable direction. The movable direction is, for example, information indicating an angle based on the vertical direction.

The information acquisition unit 231 acquires reaction force information indicating the relationship between the movable displacement amount and the reaction force generated by the selected object in association with the positions of each of the plurality of objects. The position of the object is represented, for example, by polar coordinates with respect to the central position when the object is placed in a predetermined orientation. The information acquisition unit 231 obtains, for example, reaction force information corresponding to the upper side of the object, reaction force information corresponding to the right side of the object, reaction force information corresponding to the left side of the object, reaction force information corresponding to the front side of the object, Acquire reaction force information and the like corresponding to the rear of the object.

The reaction force information corresponding to each position of the object may be different. For example, in the case of an object that has large elasticity and viscosity in the vertical direction and small elasticity and viscosity in the horizontal direction, the elastic coefficient and viscosity coefficient included in the reaction force information corresponding to the upper direction of the object, and the The corresponding elastic and viscous moduli are different.

The haptic control section 233 controls the movable section 11 to generate a movable section reaction force corresponding to the position corresponding to the orientation identified by the orientation identification section in the reaction force information and the movable section displacement amount. In the example shown in FIG. 7, the orientation identifying unit identifies that the movable unit 11 is directed to the right, and the force sense control unit 233 generates a force generated in the movable unit 11 based on the reaction force information corresponding to the right side of the object. Determine the reaction force of the movable part that

Further, the image data creation unit 235 creates deformed image data in which the shape of the position corresponding to the orientation of the haptic device 1 notified from the orientation identification unit changes according to the displacement amount of the movable part. causes the display device 3 to sequentially display the deformed image data. In the example shown in FIG. 7, an image in which the right side of the object is deformed is displayed on the display device 3, and the position of the finger is also displayed at a position corresponding to the orientation of the force sense presentation device 1. In the example shown in FIG.

By operating the force sense presentation system S in this way, the user U can confirm the feel of the object at various positions, so that the user U can confirm in detail the feel of the object whose elasticity and viscosity differ depending on the position. becomes possible.

[Second modification]
In the above description, an image in which an object is pressed with one finger is displayed on the display device 3, but the mode of the image is not limited to this. The haptic presentation device 1 may detect the number of fingers that the user U is touching the movable section 11 and notify the control device 2 of the detected number of fingers. The display control unit 236 may cause the display device 3 to display deformed image data showing a state in which the notified number of fingers are pressing an object.

[Third modification]
In the above description, the case where the displacement amount detection unit 12 detects the displacement speed of the movable part 11 is illustrated, but the displacement amount detection unit 12 does not detect the displacement speed, and the displacement amount detection unit 12 detects the movable part detected by the displacement amount detection unit 12. The force control section 233 may detect the displacement speed of the movable section 11 based on the displacement amount. The haptic control section 233 can detect the displacement speed of the movable section 11 by, for example, time-differentiating the movable section displacement amount notified from the displacement amount detection section 12 .

[Fourth Modification]
In the above description, the case where the display device 3 is a large display was exemplified, but the form of the display device 3 is arbitrary, and may be, for example, a head-mounted display. In this case, the control device 2 and the display device 3 may be integrated, and the head-mounted display may have a processor functioning as the control device 2 and a display functioning as the display device 3 .

Although the present invention has been described above using the embodiments, the technical scope of the present invention is not limited to the scope described in the above embodiments, and various modifications and changes are possible within the scope of the gist thereof. be. For example, all or part of the device can be functionally or physically distributed and integrated in arbitrary units. In addition, new embodiments resulting from arbitrary combinations of multiple embodiments are also included in the embodiments of the present invention. The effect of the new embodiment caused by the combination has the effect of the original embodiment.

1 Force sense presentation device 2 Control device 3 Display device 10 Main unit 11 Movable unit 12 Displacement amount detection unit 21 Communication unit 22 Storage unit 23 Control unit 231 Information acquisition unit 232 Selected object identification unit 233 Force control unit 234 Image data acquisition unit 235 image data creation unit 236 display control unit

Claims (6)

A haptic presentation device that presents a haptic sensation, and a control device that controls the haptic presentation device,
The haptic presentation device includes:
a movable part that generates a reaction force of the movable part corresponding to the amount of displacement caused by being pushed;
a displacement amount detection unit that detects an amount of displacement of the movable part, which is the amount of displacement of the movable part due to the pushing of the movable part;
an orientation identifying unit that identifies the orientation of the haptic presentation device;
has
The control device is
an information acquisition unit that acquires reaction force information indicating the relationship between the position of each of the plurality of objects, the amount of displacement of the movable portion, and the reaction force of the movable portion generated in the movable portion, in association with a plurality of objects;
a selected object identification unit that identifies one selected object selected from the plurality of objects;
In the reaction force information associated with the selected object, a position on the selected object corresponding to the orientation of the haptic presentation device specified by the orientation specifying unit, and displacement of the movable portion notified from the haptic presentation device a haptic control unit that controls the movable part to generate the movable part reaction force associated with a quantity ;
an image data acquisition unit that acquires object image data that is image data of the selected object;
an image data creation unit that creates deformed image data by deforming the object image data based on the displacement amount of the movable part;
a display control unit for displaying the deformed image data on a display device;
A haptic presentation system. The image data acquisition unit acquires a plurality of deformed image data corresponding to each of the plurality of movable portion displacement amounts,
The image data creation unit selects the deformed image data corresponding to the movable portion displacement amount from the plurality of deformed image data.
The haptic presentation system according to claim 1. The display control unit causes the display device to sequentially display the deformed image data having different shapes in accordance with a change in the displacement amount of the movable portion.
The force sense presentation system according to claim 1 or 2. The information acquisition unit acquires the reaction force information indicating the movable part displacement amount and the elasticity and viscosity of the selected object in association with a plurality of objects.
The force sense presentation system according to any one of claims 1 to 3. A control device for controlling a movable part of a haptic presentation device that presents a haptic,
In association with a plurality of objects , a position of each of the plurality of objects; an information acquisition unit that acquires reaction force information indicating a relationship with the internal reaction force;
a selected object identification unit that identifies one selected object selected from the plurality of objects;
In the reaction force information associated with the selected object, a position on the selected object corresponding to the direction of the haptic presentation device specified by the haptic presentation device, and a movable portion displacement amount acquired from the haptic presentation device and a haptic control section that controls the movable section so as to generate the movable section reaction force associated with,
an image data acquisition unit that acquires object image data that is image data of the selected object;
an image data creation unit that creates deformed image data by deforming the object image data based on the displacement amount of the movable part;
a display control unit for displaying the deformed image data on a display device;
A control device having a processor included in a control device that controls a movable part of a haptic presentation device that presents a haptic sensation;
In association with a plurality of objects , a position of each of the plurality of objects; an information acquisition unit that acquires reaction force information indicating the relationship with the internal reaction force;
a selected object identification unit that identifies one selected object selected from the plurality of objects;
In the reaction force information associated with the selected object, a position on the selected object corresponding to the direction of the haptic presentation device specified by the haptic presentation device, and a movable portion displacement amount acquired from the haptic presentation device and a haptic control section that controls the movable section so as to generate the movable section reaction force associated with
an image data acquisition unit that acquires object image data that is image data of the selected object;
an image data creating unit for creating deformed image data by deforming the object image data based on the displacement amount of the movable part; and a display control unit for displaying the deformed image data on a display device;
A program to function as

Images