CN108161994B - Multi-modal touch sensing device - Google Patents

Abstract

The invention provides a multi-modal touch sensing device, and belongs to the technical field of robot sensors. The device comprises a shell, wherein an image acquisition camera, a multi-array pressure sensor, a plurality of supporting structure assemblies and a transparent elastic body are arranged in the shell, and the upper part of the transparent elastic body protrudes out of the top of the shell; the image acquisition camera is fixedly connected with the bottom of the shell through a camera supporting plate; the multi-array pressure sensor is positioned above the image acquisition camera and supported by the bottom of the shell, the multi-array pressure sensor is provided with a light transmission hole, and the upper surface of the multi-array pressure sensor is provided with a transparent supporting block for supporting the transparent elastic body; the periphery of the lower part of the transparent elastomer is provided with uniformly distributed light emitters; the transparent elastomer is provided with temperature sensors at the periphery protruding out of the top of the shell. The invention integrates the measurement of multi-mode information such as stress, texture, temperature and the like in the same sensing device, and has the advantages of simple and compact structure, high measurement precision, repeated use, wide application range and the like.

Description

A multimodal tactile sensing device

technical field

The invention relates to a multimodal tactile sensing device, which belongs to the technical field of robot sensors.

Background technique

The robot dexterous hand needs to be able to perform various operations such as grasping, moving, pinching or fixing objects. During the operation, the tactile feedback of the dexterous hand must be used to achieve precise control of the operating force to avoid damage to the manipulated object. There are many kinds of conventional dexterous hand sensors, including piezoresistive, capacitive, photoelectric, etc., that is, the strength is judged by the change of resistance, capacitance, and light intensity under the action force. However, these sensors have some shortcomings. For example, sensors based on voltage or capacitance changes are sensitive to noise and temperature changes, and can only detect vertical force, but cannot detect shear force.

To overcome the above problems, vision-based tactile sensing devices were proposed in the early 21st century. The original installation was developed by the tachi team at the University of Tokyo. The installation includes cameras, transparent blocks and transparent elastomers arranged in sequence from bottom to top. The interior of the transparent elastomer is vertically arranged with two rows of different colors (red and blue). and the upper and lower rows of marking points are staggered; the force of the device is solved according to the displacement of the marking points in the elastic body obtained by the bottom camera. Based on the same principle, his team designed a vision-based tactile sensing device placed on the tip of a dexterous finger in 2010, and applied the device to a robot dexterous hand, but it was limited to the measurement of the force on the fingers of the robot dexterous hand. In 2014, Edward H. Adelson's team at MIT designed a sensing device based on Gelsight (elastomer material) that can only obtain the surface texture of an object. Then the team added a denser layer of irregular black marking points to the Gelsight material. Due to the irregularity and denseness of the marking points, the texture information was partially lost. After processing the displacement changes of the marking points, the sensing device was obtained. The information such as force and slippage can not be obtained, but the texture information of the object cannot be obtained.

To sum up, today's vision-based tactile sensing devices can only obtain single-modal information, the collected information is too single, and the robot obtains less relevant information, which seriously reduces the intelligence of the robot's perception of actions.

SUMMARY OF THE INVENTION

The purpose of the present invention is to solve the problem that the existing tactile sensor can only obtain single-modal information, so as to provide a multi-modal tactile sensing device. The device can obtain various modal information such as three-dimensional contact force, temperature and texture of the surface of the contacting object. By collecting these information, the contact state of the robot hand and the object and the surface properties of the contacting object can be obtained, which can improve the perception of the robot hand for the subsequent control terminal. The precision and stability of the operation provide the basis.

The technical scheme that the present invention takes to solve the above problems is:

A multimodal tactile sensing device is characterized in that it comprises a casing, an image acquisition camera, a multi-array pressure sensor, a plurality of supporting structure components and a transparent elastomer are arranged inside the casing, and the upper part of the transparent elastomer protrudes out of the casing The top; wherein, the image acquisition camera is fixedly connected with the bottom of the casing through the camera support plate; the multi-array pressure sensor is located above the image acquisition camera and supported by the shoulders fixed on the four corners of the bottom of the casing, the multi-array pressure sensor There is a light-transmitting hole; the upper surface of the multi-array pressure sensor is provided with a transparent support block for supporting the transparent elastic body, and the upper periphery of the transparent support block is limited by the middle fixing plate, which is supported on the side wall of the casing ; The lower part of the transparent elastic body is provided with evenly distributed light emitters, and the light emitter is supported by the middle fixing plate; the transparent elastic body is protruded from the top of the casing and is provided with a temperature sensor, and the temperature sensor is embedded On the top of the casing; the lens of the image acquisition camera, the light-transmitting holes of the multi-array pressure sensor, the transparent support block and the transparent elastic body are all arranged on the same optical axis.

Optionally, the transparent elastomer is made of a transparent material polydimethylsiloxane (PDMS), and the transparent elastomer protrudes from the upper surface of the top cover by sputtering metal aluminum with a thickness of 2000A first and then laser spotting. A 8*8 marking point array is set up in this way, and then a copper film with a thickness of 2000A is sputtered on the marking point array and other areas on the upper surface of the transparent elastomer to obtain the surface texture of the contact object.

Features and beneficial effects of the present invention:

1. This device breaks the status quo of single-modal information collection of traditional tactile sensing devices. It can integrate the measurement of force, texture, temperature and other information in the same sensing device, realize the measurement of multi-modal information, and synthesize these information. To recognize and grasp an object, make the robot more intelligent and humanized;

2. The three-dimensional contact force can be detected through the coordination of the multi-array pressure sensor in the tactile sensing device and the marking point of the transparent elastic body;

3. The transparent elastic body in the tactile sensing device can obtain high-precision contact texture information. The copper mold sputtered on the upper surface of the transparent elastic body has the following advantages: 1) It is convenient to identify the marking point; 2) It can protect the marking point from being Wear; 3) It has strong ductility and anti-reflection, so as to obtain a very fine texture of the object, and make the transparent elastomer less prone to cracks;

4. The tactile sensing device is simple and compact in structure, low in production cost and reusable;

5. The hardness of the transparent elastomer can be adjusted according to the task requirements, so that the tactile sensing device has the required sensitivity;

6. The tactile sensing device can be applied to the detection of tactile signals in robot hands and other complex environmental scenarios.

Description of drawings

FIG. 1 is a schematic diagram of the overall structure of the multimodal tactile sensing device of the present invention.

FIG. 2 is a schematic diagram of a marked point on a marked elastic body in the vision-based tactile sensor of the present invention.

3 is an image of the elastic body in the vision-based tactile sensor of the present invention in contact with the surface of the towel and deforming.

FIG. 4 is a schematic structural diagram of a multi-array pressure sensor body according to an embodiment of the present invention.

FIG. 5 is a top view of FIG. 4 .

Among them, 1. Top cover, 2. Transparent elastomer, 3. Temperature sensor, 4. Light emitter, 5. Middle fixing block, 6. Transparent support block, 7. U-shaped base, 8. Camera support plate, 9. Image Acquisition camera, 10. Multi-array pressure sensor.

Detailed ways

A multimodal tactile sensing device proposed by the present invention is described in detail as follows with reference to the accompanying drawings and embodiments:

The overall structure of a multi-modal tactile sensing device proposed by the present invention is shown in Figure 1. The tactile sensing device is worn at the fingertips of the robot's dexterous hand, and includes a shell composed of a U-shaped base 7 and a top cover 1. An image acquisition camera 9, a multi-array pressure sensor 10, a plurality of supporting structure components and a transparent elastic body 2 are arranged inside the casing, and the upper part of the transparent elastic body 2 protrudes from the top cover 1; wherein, the image acquisition camera 9 passes through the camera support plate 8 is fixedly connected with the base 7, the multi-array pressure sensor 10 is located above the image acquisition camera 9 and supported by the shoulders 7-1 fixed on the four corners of the base 7, and the multi-array pressure sensor 10 is structurally provided with light-transmitting holes 10- 1. The upper surface of the multi-array pressure sensor 10 is provided with a transparent support block 6 for supporting the transparent elastic body 2. The upper periphery of the transparent support block 6 is limited by the middle fixing plate 5, and the middle fixing plate 5 is supported on the side wall of the base 7 The lower part of the transparent elastic body 2 is provided with uniformly distributed light emitters 4, and the light emitters 4 are supported by the middle fixing plate 5; The temperature sensor 3 is embedded on the surface of the top cover 1; the lens of the image acquisition camera, the light-transmitting hole 10-1 of the multi-array pressure sensor, the transparent support block 6 and the transparent elastic body 2 are all arranged on the same optical axis, and the transparent elastic body and the image The distance of the acquisition camera meets the imaging requirements.

The specific structure of the multimodal tactile sensing device will be described in detail below in conjunction with the accompanying drawings and embodiments:

The transparent elastomer 2 is used to imitate the touch and force of human fingers, and its top view is shown in Figure 2. The transparent elastomer is made of a transparent material polydimethylsiloxane (PDMS). The basic components of PDMS and curing agent are used to obtain elastomers with different softness (the specific manufacturing process is known in the art), and the elastomer has low elastic modulus, high repeatability and metal adhesion; the transparent elastomer protrudes from the The upper surface of the top cover 1 is set with an 8*8 marking point array 2-1 by first sputtering metal aluminum with a thickness of 2000A (10A=1nm) and then laser spotting, a total of 64 marking points, and then on the marking point array and On other areas on the upper surface of the transparent elastomer, sputter a layer of metal film with a thickness of 2000A (a metal with good ductility such as copper can be used) 2-2 to obtain the surface texture of the contact object. The transparent elastomer has the advantages of convenient and quick production. , easy to modify, and high spatial resolution. The role of the sputtered copper film is as follows: 1) The color difference between the metal copper and the sputtered metal aluminum marking point is obvious, so that the marking point is easier to identify; 2) The metal aluminum marking point is protected and prevented from being worn; 3) It has strong The ductility and anti-reflection properties are excellent, so as to obtain a very fine texture of the object, and make the transparent elastomer less prone to cracks. When the transparent elastic body is stressed, the corresponding displacement changes will occur at each marked point, so that the transmission of the force information of the sensing device can be better completed. The transparent elastic body in this embodiment is a rectangular parallelepiped with a length of 13 mm, a width of 13 mm and a height of 8 mm, the diameter of each marking point is 0.4 mm, and the spacing between adjacent marking points is 1 mm. Figure 3 shows the texture of the towel surface obtained by the elastic body in this embodiment. From the figure, the fine texture of the towel cloth and the direction of the texture can be clearly seen. The texture obtained by this device is even more subtle than the human eye can observe the texture. degree.

The image acquisition camera 9 is used to obtain the position of each marked point on the transparent elastic body 2 and the image of the metal film on the surface of the transparent elastic body through the transparent support block 6. In this embodiment, a USB camera module is used, and the focal length is 20-25 mm. 27mm*7mm*6mm, generating 30 frames per second, the image capture camera 9 sends the captured images to the external control terminal in real time through the USB interface.

The light-emitting device 4 adopts LED lamps and a matching circuit board. In this embodiment, 8 patch-type white LED lamps are evenly arranged around the transparent elastic body 2 and the total voltage does not exceed 4V. The light emitter is responsible for providing stable and uniform illumination to the transparent elastomer 2 to avoid detection deviation caused by changes in the brightness of natural light.

The multi-array pressure sensor adopts a capacitive multi-array sensor, including a PCB circuit board (the PCB circuit board is not shown in Figure 4) and the multi-array pressure sensor body fixed on the PCB circuit board, the PCB circuit board and the U-shaped base 7 solid connection. The structure of the multi-array pressure sensor body is shown in Figure 4, including an upper sensing electrode plate I-1, a lower sensing electrode plate I-2, and several flexible microneedles I-3 fixed between the upper and lower sensing electrode plates (for the purpose of To illustrate the arrangement of the flexible microneedles more clearly, Fig. 4 cuts the flexible microneedles from the middle and is divided into upper and lower sections. In practice, the flexible microneedles connecting the upper and lower sensing electrode plates are a whole). Among them, the structure of the upper and lower sensing plates is the same, and 25 tactile sensing points are integrated in a 1cm ² unit. The 25 tactile sensing points include a main sensor for detecting the pressure on the middle of the transparent elastic body 2 Points 10-2 and 24 are secondary sensing points (10-3 to 10-26) used to detect the pressure around the transparent elastic body 2. The primary sensing point 10-2 is a 6mm radius with a transparent Circular metal plate with light hole 10-1 (the diameter of the light-transmitting hole is 5mm); 24 secondary sensing points are concentric fan-shaped metal plates evenly arranged around the main sensing point, that is, there is a secondary sensing point every 15° counterclockwise The sensing points are numbered 10-3 to 10-26 in sequence, as shown in Figure 5, all sensing points are connected to the digital-analog interface of the external control terminal. The flexible microneedle is made of polydimethylsiloxane (PDMS) through a conventional overmolding process. PDMS has good elasticity. When the sensing electrode plate is pressed, the flexible microneedle is bent, and when the sensing electrode is pressed, the flexible microneedle is bent. When there is no pressure on the plate, the flexible microneedle will quickly return to its original state, so that the distance between the upper and lower sensing plates can be restored to the original level. Using this flexible microneedle, the sensitivity of capacitive multi-array sensor can be improved. Both the upper and lower induction stage plates are wrapped by polydimethylsiloxane flexible substrate. Using this material to wrap the induction plate can shield the electrostatic interference and prevent the oxidation of the induction plate at the same time. The upper sensing electrode plate and the lower sensing electrode plate are bonded by oxygen plasma filling treatment, and the upper and lower sensing electrode plates are bonded by the chemical bond between molecules, which ensures the scalability of the capacitive multi-array sensor. and softness. The working principle of the capacitive multi-array sensor is as follows: a plate capacitor is formed between the upper sensing electrode plate and the lower sensing electrode plate. When the pressure value applied to the multi-array sensor changes, the deformation of the flexible micro-needles makes the upper , The distance between the lower sensing plates changes, which leads to the change of the capacitance value. The device infers the size of the tactile force by monitoring the change of the capacitance value of the plate capacitor.

The temperature sensor is a contact temperature sensor. When the manipulator grabs, the elastic body is forced to sag, and the temperature sensor touches the object, which can quickly measure the temperature of the surface of the object. This embodiment uses a temperature sensor with model LM71, including a 5-pin SOT-23 sub-packaging box and a 6-pin non-pullback package. The measured temperature range is -40°C—150°C, and the temperature accuracy is: -10 The maximum error is ±1.5℃ at ℃—65℃, and the maximum error is +3/-2℃ when the temperature is -40℃—150℃.

The support structure assembly includes a camera support plate 8, a transparent support block 6 and a middle fixing plate 5; wherein, the camera support plate 8 is used to fix and position the image capture camera 9, and the camera support plate is fixed on the base 7 of the casing by screws; The lower surface of the transparent support block 6 is attached to the upper surface of the multi-array pressure sensor 10, the upper surface of the transparent support block 6 is attached to the lower surface of the transparent elastic body 2, and the transparent elastic body is supported by the transparent support block 6, and at the same time The light generated by the illuminator is made more uniform, which is convenient to improve the clarity of the image captured by the camera, thereby ensuring the quality of the image captured; the middle fixing plate 5 is used to fix the transparent support block 6 and support the illuminator 4 . The transparent support block 6 in this embodiment is made of acrylic plate material, which has a high light transmittance of more than 92%, and the rest of the support structure components are made of resin by 3D printing, which has high processing efficiency and low cost.

The casing is used for accommodating and supporting the components of the embodiments of the present invention, and may adopt different shapes according to the requirements of actual use scenarios. The shell of this embodiment is a closed square box, which is made of resin material by 3D printing, and the processing cost is low. The length, width and height are 49mm, 24mm, and 23mm.

The working process of the multimodal tactile sensing device proposed by the present invention is described below in conjunction with the accompanying drawings:

When the transparent elastic body 2 of the sensor is in contact with the object, the pressure in the vertical direction and the shearing force in the horizontal direction will cause the transparent elastic body to deform, and the marking points on the transparent elastic body will be displaced accordingly. The illuminator (LED lamp) 4 is stimulated by the control circuit to provide a stable light source, the image acquisition camera 9 captures the image information before and after the displacement of the marker point through the light-transmitting support block 6, and the displacement information of the marker point is calculated by the external control terminal, The shear force in the horizontal direction on the surface of the object is obtained (the specific calculation process can be realized by using conventional techniques in the field, and does not belong to the protection category of the present invention); the transparent elastic body 2 will transmit the force to the multi-array pressure sensor 10 after being stressed For pressure, the multi-array sensor 10 outputs the force in the vertical direction to the external control terminal. When the transparent elastic body 2 is recessed, the temperature sensor 3 contacts the surface of the object, and outputs the temperature of the surface of the object to the external control terminal. Then, these forces, textures, and temperature information are integrated and displayed on the external control terminal.

Claims (7)

1. A multimodal tactile sensing device, characterized in that it comprises a casing, and the casing is provided with an image acquisition camera, a multi-array pressure sensor, a plurality of supporting structure components and a transparent elastic body, and the upper part of the transparent elastic body protrudes in the casing; wherein, the image acquisition camera is fixedly connected to the bottom of the casing through a camera support plate; the multi-array pressure sensor is located above the image acquisition camera and supported by the shoulders fixed on the four corners of the bottom of the casing, the multi-array pressure sensors The array pressure sensor is provided with a light-transmitting hole; the upper surface of the multi-array pressure sensor is provided with a transparent support block for supporting the transparent elastic body, and the upper periphery of the transparent support block is limited by a middle fixing plate, which is supported on the casing on the side wall; the lower part of the transparent elastic body is provided with evenly distributed light emitters, and the light emitter is supported by the middle fixing plate; the transparent elastic body is protruded from the top of the casing with a temperature sensor, and the temperature The sensor is embedded on the top of the casing; the lens of the image acquisition camera, the light-transmitting hole of the multi-array pressure sensor, the transparent support block and the transparent elastic body are all arranged on the same optical axis; The transparent elastomer is made of a transparent material polydimethylsiloxane (PDMS), and the transparent elastomer protrudes from the upper surface of the casing to set 8*8 by first sputtering metal aluminum with a thickness of 2000A and then laser spotting Then, a copper film with a thickness of 2000A is sputtered on the marking point array and other areas on the upper surface of the transparent elastomer to obtain the surface texture of the contact object. 2 . The multimodal tactile sensing device according to claim 1 , wherein the transparent elastic body is a rectangular parallelepiped, 13 mm long, 13 mm wide and 8 mm high, and the diameter of each marking point is 0.4 mm, and the distance between adjacent marking points is 0.4 mm. 3 . The pitch is 1mm. 3 . The multimodal tactile sensing device according to claim 1 , wherein the focal length of the image acquisition camera is 20-25 mm. 4 . 4 . The multimodal tactile sensing device according to claim 1 , wherein the light emitter adopts an LED lamp and a matching circuit board, and the total voltage of the light emitter does not exceed 4V. 5 . 5. The multimodal tactile sensing device according to claim 1, wherein the multi-array pressure sensor comprises a PCB circuit board and a multi-array pressure sensor body fixed on the PCB circuit board, the PCB circuit The plate is fixedly connected with the bottom of the casing; wherein, the multi-array pressure sensor body includes an upper sensing electrode plate, a lower sensing electrode plate, and a number of flexible micro-needles fixed between the upper and lower sensing electrode plates. The upper and lower sensing plates have the same structure, and 25 tactile sensing points are integrated in a 1cm ² unit, and the 25 tactile sensing points include 1 for detecting the middle part of the transparent elastomer The main sensing point of the pressure and 24 secondary sensing points used to detect the pressure around the transparent elastomer. The main sensing point is a circular metal with a radius of 0.4cm and the light-transmitting hole in the center. The 24 secondary sensing points are concentric fan-shaped metal plates evenly distributed around the main sensing point, and all sensing points are connected to the digital-analog interface of the external control terminal. 6 . The multimodal tactile sensing device according to claim 1 , wherein the temperature sensor is a contact temperature sensor. 7 . 7 . The multimodal tactile sensing device according to claim 1 , wherein the casing is a closed square box, made of resin material by 3D printing, and the length, width and height are 49mm, 24mm, 23mm.

Images