CN111721451A - Image-based tactile sensing method, miniaturized device and device fabrication method - Google Patents

Abstract

Embodiments of the present invention provide an image-based tactile sensing method, a miniaturized device, and a method for preparing the device, including: using a light-emitting source to project light toward the top of the micro-pillar, and the light is emitted through the micro-pillar and the substrate; the micro-pillar is located on the substrate A flexible transparent micro-pillar array is formed; the image sensor disposed at the bottom of the flexible transparent micro-pillar array is used to receive the emitted light, and a micro-pillar deformation image is generated; the micro-pillar deformation image is input into the decoupling network model, and the output of the applied three-dimensional force Three-dimensional perception data; among them, the decoupling network model is obtained after training based on the sample spot image of the flexible transparent micro-pillar array subjected to standard pressure and the corresponding identification label. In the embodiment of the present invention, by integrating the light source in the pressure sensor, the bending and deformation of the micro-pillar can be clearly captured, and the intelligent algorithm is used to complete the decoupling of the three-dimensional force based on the deformation image, thereby realizing the miniaturization and height of the sensor. Integration and precision.

Description

Image-based tactile sensing method, miniaturized device and device fabrication method

technical field

The invention relates to the technical field of intelligent perception and the field of flexible tactile sensors, and more particularly, to an image-based tactile sensing method, a miniaturized device and a device preparation method.

Background technique

Nowadays, the demand for tactile perception in the field of robot application is increasing. When robots can accurately complete various delicate and difficult tasks, they must be able to detect spatial multi-dimensional forces, and an integrated and miniaturized tactile sensing method is required. As a support, while taking into account the three-dimensional force measurement, it can be as flexible and highly integrated as a real human hand. Research on tactile sensing methods is imminent.

At present, the research on tactile sensing methods is mainly based on strain type, piezoelectric type, capacitive type and piezoresistive type. Among them, strain sensors generally have poor flexibility and are not suitable for the flexible skin of robots. Piezoelectric sensors work stably and respond sensitively to external forces. However, due to their large internal resistance, they are only suitable for measuring limited dynamic forces, and almost impossible to measure static forces. Therefore, there are certain difficulties in the decomposition of three-dimensional forces. For capacitive sensors, due to the limitation of volume, the sensing capacitance is often small, which makes the measurement susceptible to strong interference by parasitic capacitance, and the precise measurement circuit is also complicated, which greatly limits its practical application. Resistive sensors are relatively difficult to decouple due to many internal cross points, and the array sensor leads are complicated, which limits the application of resistive sensors.

Compared with electronic components such as capacitors and resistors, the image-based tactile sensing method can effectively reduce the volume of the measurement unit and the number of sensor connections, and can better realize the miniaturization and high integration of the sensor. The detection sensitivity of the force is high, the three-dimensional force can be identified very accurately, and the temperature can be measured at the same time, which has far-reaching significance for the realization of artificial touch. However, the existing image-based tactile sensing methods have poor imaging effects, resulting in low measurement accuracy and are easily limited by the application environment.

In view of this, whether to provide a flexible three-dimensional force detection method and device that is miniaturized, has high detection sensitivity, is less restricted by the application environment, and is convenient for decoupling has far-reaching significance for the realization of artificial touch.

SUMMARY OF THE INVENTION

In order to solve the above problems, embodiments of the present invention provide an image-based tactile sensing method, a miniaturized device, and a device manufacturing method that overcome the above problems or at least partially solve the above problems.

In a first aspect, an embodiment of the present invention provides an image-based tactile sensing method, including: using a light-emitting source to project light toward the top of a micro-pillar, and the light is emitted through the micro-pillar and the substrate; the micro-pillar is located on the substrate to form a flexible A transparent micro-pillar array; an image sensor disposed at the bottom of the flexible transparent micro-pillar array is used to receive the emitted light, and a micro-pillar deformation image is generated; the micro-pillar deformation image is input into the decoupling network model, and the output applied on the flexible transparent Three-dimensional perception data of three-dimensional force; among them, the decoupling network model is obtained after training based on the sample spot image and the corresponding identification label of the flexible transparent micro-pillar array subjected to standard pressure.

Optionally, before using the light-emitting source to project light toward the top of the micro-pillar, the method further includes: setting the light-emitting source as a light-emitting film; spin-coating the light-emitting film on the top of the micro-pillar; removing the bottom of the flexible transparent micro-pillar array; The other parts of the micropillars except the top part are coated with a light-blocking material layer.

Alternatively, before using the light-emitting source to project light toward the top of the micro-pillars, the method may further include: setting the light-emitting source as a light-emitting film; coating a light-blocking material layer on the top of each micro-pillar; spin-coating the light-emitting film on the light-blocking material layer.

Optionally, the above-mentioned micro-columns are widened micro-columns, and the diameter of the widened micro-columns is 500 μm-1 cm.

Optionally, before using the light-emitting source to project light toward the top of the micro-columns, the method may further include: setting the light-emitting source as a fluorescent coating; coating the top of each micro-column with a fluorescent coating; The other parts of the column array except the bottom are coated with a light blocking material layer.

Alternatively, before using the light-emitting source to project light toward the top of the micro-pillars, the method may further include: coating a light-blocking material layer on the top of each micro-pillar; disposing a grating layer between the light-emitting source and the light-blocking material layer , the grating layer is used to make the light incident vertically to the top of the micro-column; the light-emitting source is set as a light-emitting film or a fluorescent coating.

Optionally, a variable color developing layer is added on the inner side of the light-blocking material layer close to the micro-pillars.

In a second aspect, embodiments of the present invention provide an image-based miniaturized tactile sensing device, including but not limited to: a light-emitting source, a flexible transparent micro-pillar array, an image sensor, and an image processing unit; The light is projected in the top direction, and the light is emitted through the micro-pillars and the substrate; the micro-pillars are located on the substrate and integrally formed to form a flexible transparent micro-pillar array; the image sensor is placed at the bottom of the flexible transparent micro-pillar array to receive the emitted light and generate micro-pillars. Column deformation image; the image processing unit pre-stores a decoupling network model for inputting the micro-column deformation image to the decoupling network model, and outputs the 3-D perception data of the 3-D force exerted on the flexible transparent micro-column array; wherein the decoupling The network model is obtained after training based on the sample spot image of the flexible transparent micro-pillar array subjected to standard pressure and the corresponding identification label.

In a third aspect, embodiments of the present invention provide a method for preparing an image-based miniaturized tactile sensing device, including but not limited to the following steps: preparing a flexible transparent micro-pillar array, coating a variable display on the flexible transparent micro-pillar array a color layer and coating a light blocking material layer on the variable color developing layer.

Wherein, preparing the flexible transparent micro-pillar array includes: spin-coating photoresist on the inner surface of the preparation container; exposing and developing the preparation container; setting the preset pattern of the flexible transparent micro-pillar array in the preparation container On the photoresist on the surface, make a mold for the flexible transparent micro-pillar array; after the prepolymer for making the flexible transparent micro-pillars in the entire array of flexible transparent micro-pillars and the silicone oil are uniformly mixed according to the first set ratio, pour it on the mold, and put it on the mold. After heating and curing, it is poured into a flexible transparent micro-pillar array;

Coating a variable color developing layer on the flexible transparent micro-pillar array, including: bonding a glass sheet on the top of the micro-pillars; uniformly mixing the liquid variable color developing material and the prepolymer in a second set ratio to generate a mixed color developing material liquid; after injecting the color-developing material mixed liquid into the gap between the micro-columns of the flexible transparent micro-column array, blow out the excess color-developing material mixed liquid with gas; after heating to solidify the color-developing material mixed liquid, remove the glass sheet;

The coating of the light-blocking material layer on the variable color development layer includes: re-bonding the glass sheet on the top of the micro-pillar; uniformly mixing the liquid light-blocking material and the prepolymer at a third set ratio to generate a light-blocking material mixed liquid; After injecting the light-blocking material mixed liquid into the gaps between the micro-pillars of the flexible transparent micro-pillar array, the excess light-blocking material mixed liquid is blown out with gas; after heating to solidify the light-blocking material mixed liquid, the glass sheet is removed.

The image-based tactile sensing method, the miniaturized device and the device preparation method provided by the embodiments of the present invention can clearly capture the bending and deformation of the micro-pillar by integrating the light source in the pressure sensor, and use an intelligent algorithm based on the deformation Image, completes the decoupling of three-dimensional force, and realizes the miniaturization, high integration and precision of the sensor.

Description of drawings

In order to illustrate the embodiments of the present invention or the technical solutions in the prior art more clearly, the following briefly introduces the accompanying drawings that are required in the description of the embodiments or the prior art. Obviously, the drawings in the following description are only some embodiments of the present invention, and for those of ordinary skill in the art, other drawings can also be obtained from these drawings without creative effort.

1 is a schematic flowchart of an image-based tactile sensing method provided by an embodiment of the present invention;

2 is a schematic diagram of an image-based miniaturized tactile sensing device and a working principle according to an embodiment of the present invention;

3 is a flow chart of decoupling of a three-dimensional force measurement method in an image-based tactile sensing method provided by an embodiment of the present invention;

4 is a schematic diagram of another image-based miniaturized tactile sensing device and a working principle according to an embodiment of the present invention;

5 is a schematic diagram of another image-based miniaturized tactile sensing device and a working principle according to an embodiment of the present invention;

FIG. 6 is a schematic diagram of yet another image-based miniaturized tactile sensing device and a working principle according to an embodiment of the present invention;

7 is a schematic diagram of a model of an image-based miniaturized tactile sensing device according to an embodiment of the present invention;

FIG. 8 is a process flow diagram for preparing an image-based miniaturized tactile sensing device provided by an embodiment of the present invention;

FIG. 9 is a process flow diagram of preparing a flexible transparent micro-pillar array in an image-based miniaturized tactile sensing device provided by an embodiment of the present invention.

Detailed ways

In order to make the purposes, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly described below with reference to the drawings in the embodiments of the present invention. Obviously, the described embodiments are the Some, but not all, embodiments are disclosed. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

An embodiment of the present invention provides an image-based tactile sensing method, as shown in FIG. 1 , the method includes but is not limited to the following steps:

Step S1, using a light-emitting source to project light toward the top of the micro-pillars, and the light is emitted through the micro-pillars and the substrate; the micro-pillars are positioned on the substrate to form a flexible transparent micro-pillar array;

Step S2, using the image sensor disposed at the bottom of the flexible transparent micro-pillar array to receive the emitted light to generate a micro-pillar deformation image;

Step S3, input the micro-column deformation image into the decoupling network model, and output the three-dimensional perception data of the three-dimensional force exerted on the flexible transparent micro-column array; wherein, the decoupling network model is based on the flexible transparent micro-column array subjected to standard three-dimensional force action The sample spot images and corresponding identification labels are obtained after training.

Specifically, the image-based tactile sensing method provided by the embodiment of the present invention uses a pre-trained decoupling network according to the obtained micro-pillar deformation image generated by the flexible transparent micro-pillar array under the action of the three-dimensional force exerted on it. The model performs feature analysis on the micro-column deformation image to obtain the three-dimensional perception data of the three-dimensional force.

Because the acquisition of micro-column deformation images is affected by many factors that interfere with the external environment, especially in the working environment with low light, images that can clearly reflect the micro-column deformation are often not obtained, or even cannot work normally at all. Therefore, in the image-based tactile sensing method provided by the embodiment of the present invention, a light source is added, and the light source is used to project light from a direction perpendicular to the plane where the top of the micro-pillar is located. If there is no occlusion on the entire optical path from the top of the micro-pillar to the substrate, the light will form a complete bright spot image at the bottom of the flexible transparent micro-pillar array; there is partial occlusion on the entire optical path from the top of the micro-pillar to the substrate In the case of , a incomplete bright spot image is formed at the bottom of the flexible transparent micro-pillar array; if the entire light path from the top of the micro-pillar to the substrate is completely blocked, a shadow image is formed at the bottom of the flexible transparent micro-pillar array.

Further, an image sensor (such as CCD or CMOS, etc.) disposed at the bottom of the flexible transparent micro-pillar array is used to acquire image information of the transmitted light, and the external three-dimensional force of the flexible transparent micro-pillar array is generated according to the acquired image. image of micropillar deformation.

Finally, the micro-column deformation image is input into the pre-trained decoupling network model to decouple the 3D force, and the 3D perception data of the 3D force is obtained.

Alternatively, in this embodiment of the present invention, whether to add a light-blocking material layer at the top of the micro-pillars can be selected to determine whether the formed micro-pillar deformation image is realized according to the bright spot image or the shadow image.

If the light-blocking material layer is not provided at the top of the micro-pillars, the light can pass through the flexible and transparent micro-pillars and the base to generate a corresponding bright spot image. Bright spot image to construct micro-column deformation image.

Correspondingly, if a light-blocking material layer is provided at the top of the micro-column, the light will not pass through the top of the micro-column, nor can it be completely scattered, and a shadow will be cast under the micro-column to form a shadow image. At the bottom of the micro-pillar array, the shadow images of the light passing through other parts except the top of the micro-pillars are collected to construct the deformation image of the micro-pillars.

As shown in FIG. 2 , the flexible transparent micro-pillar array may include a plurality of flexible transparent micro-pillars. All the flexible and transparent micro-pillars are arranged in an array according to actual application requirements to form the flexible transparent micro-pillar array. The embodiment of the present invention is not flexible and transparent. The arrangement of the micro-pillars is specifically limited.

Among them, the shape of the micro-pillars is not limited to circular micro-pillars, triangular micro-pillars, polygonal micro-pillars, etc.; the diameter of the micro-pillars can be set from 1 μm to several cm, etc.; the height of the micro-pillars can be adjusted according to the diameter of the micro-pillars, generally It is 0.1 times to 10 times the height of the micro-pillars, and the spacing of the micro-pillars is more than 0.1 times the height of the micro-pillars.

The light-emitting source may be a flexible film that can emit light, such as a luminous film, an LED film, a fluorescent coating, etc., which is not specifically limited in this embodiment.

When part of the flexible transparent micropillars in the flexible transparent micropillar array are deformed under the action of three-dimensional force, the obtained micropillar deformation image changes accordingly.

In the method of this embodiment, the flexible transparent micro-columns will deform correspondingly under the action of pressure. Specifically, when subjected to horizontal force, the flexible transparent micro-columns will bend in the direction of the horizontal force, and the greater the horizontal force When the vertical force is applied, the vertical projection radius of the flexible transparent micro-pillar will increase accordingly, and the vertical projection radius will be larger when the vertical force is greater.

Further, within the elastic range of the flexible transparent micropillars, the magnitude of the horizontal force is linearly proportional to the bending degree of the flexible transparent micropillars; the magnitude of the vertical force is also linearly proportional to the change of the vertical projection radius of the flexible transparent micropillars. If a horizontal force and a vertical force are applied to the flexible transparent micropillar at the same time, the flexible transparent micropillar will bend and increase the vertical projection radius at the same time. The embodiment of the present invention measures the three-dimensional force based on the deformation of the flexible transparent micro-pillar under force.

By acquiring the deformation image of the micro-column after the flexible transparent micro-column array is subjected to three-dimensional force, the embodiment of the present invention does not specifically limit how to acquire the deformation image and the device for acquiring the deformation image.

After obtaining the deformation image of the flexible transparent micro-pillar array subjected to three-dimensional force, the obtained deformation image is input into the decoupling network model that has been trained, and the obtained image is analyzed based on the decoupling network model. Processed to output sensing data of the three-dimensional force exerted on the flexible transparent micropillar array.

Optionally, the deformation image is input into the decoupling network model, and before the perception data of the three-dimensional force is output, that is, after step S2 is performed, the deformation image of the flexible transparent micro-pillar array under the action of the three-dimensional force is obtained. , first, the deformed image is sharpened by focusing algorithm.

Specifically, after a three-dimensional force is applied to the flexible transparent micro-pillar array, the acquired deformation image of the flexible transparent micro-pillar array under the action of the three-dimensional force may be blurred due to weak defocusing. In the image-based tactile sensing method provided by the embodiment of the present invention, a focusing algorithm can be used to clear the acquired deformed image. Specifically, it can be as follows: using an out-of-focus blur image restoration algorithm, such as NAS-RIF and a method based on regular constraints, etc.; also can use intelligent zoom and other intelligent algorithms, etc., the embodiment of the present invention does not limit the focusing algorithm used,

Optionally, before inputting the deformation image to the decoupling network model, the method further includes applying a standard three-dimensional force to the flexible transparent micro-pillar array, and acquiring a sample deformation image after the flexible transparent micro-pillar array is acted by the standard three-dimensional force; The image is input into the pre-trained decoupling network model, the deformation and bending characteristics of the flexible transparent micro-pillar array are extracted, and the deformation and bending characteristics are corresponding to the perception data of the standard three-dimensional force, and the identification label is established; by changing the standard three-dimensional force direction and size, and obtain the deformation image of the flexible transparent micro-pillar array after each change; correspond each different standard three-dimensional force with the corresponding deformation image one-to-one, and complete the standard three-dimensional force on the flexible transparent micro-pillar array. After the sample deformation image and the training of the corresponding identification labels, the decoupling network model is obtained; the deformation image includes the deformation and bending features of the flexible transparent micro-pillar array.

As shown in FIG. 3 , an embodiment of the present invention provides an image-based tactile sensing method, including but not limited to the following steps:

The deformed image of the flexible transparent micro-pillar array after three-dimensional force is applied is obtained by photographing the image sensor, and the deformed image is input into the computer, and the computer is used to perform the following processing on the deformed image: Defining processing, and further extracting features such as deformation amount and curvature of the flexible transparent micro-pillar array in the sharpened deformation image, and then, according to the extracted features such as the deformation amount and the bending rate of the flexible transparent micro-pillar array. The applied 3D force is decoupled; finally, the perception data for that 3D force is output and displayed.

Further, the standard three-dimensional force can be applied to the flexible transparent micro-pillar array, and then the three-dimensional force perception data output by the computer can be obtained by the above method, and the output three-dimensional force perception data can be compared and analyzed with the standard three-dimensional force, and finally obtained. Performance parameters of flexible transparent micropillar arrays, such as sensitivity, measurement range, coupling error, etc. Further, feedback adjustment may be performed on the image-based tactile sensing method provided by the embodiment of the present invention according to the result of the comparison and analysis.

The image-based tactile sensing method provided by the embodiment of the present invention can clearly capture the bending and deformation of the micro-pillar by integrating the light source in the pressure sensor, and use an intelligent algorithm to complete the solution of the three-dimensional force based on the deformation image. The coupling realizes the miniaturization, high integration and precision of the sensor.

Based on the content of the above-mentioned embodiment, as an optional embodiment, before using the light-emitting source to project light toward the top of the micro-column, the method further includes: setting the light-emitting source as a light-emitting film; spin-coating the light-emitting film on the top of the micro-column ; Coat the light-blocking material layer on the flexible transparent micro-pillar array except the bottom and other parts of the micro-pillar except the top part.

FIG. 2 is a schematic diagram of an image-based miniaturized tactile sensing device and its working principle provided by an embodiment of the present invention. As shown in FIG. 2 , a pressure sensor constructed by using the image-based tactile sensing method provided by an embodiment of the present invention The sensor device is generally composed of three parts, including: an image sensor (COMS, CCD, etc.), a flexible transparent micro-pillar array and a light-emitting source.

Among them, in order to effectively achieve high integration and compact structure, the light source can be selected as a flexible film that can emit light, such as a luminous film, an LED film, and a fluorescent film. With this arrangement, the overall thickness of the fabricated pressure sensing device can be made not to exceed 1 cm.

Figure A in FIG. 2 shows the overall structure of the pressure sensing device, and a flexible transparent micro-pillar array is arranged above the image sensor. Alternatively, the image sensor can acquire the light transmitted from the bottom of the entire micro-pillar array. The light-emitting film is attached to the surface of the micro-pillars, so that the light emitted by the light-emitting film is vertically incident on the top of the micro-pillars.

Further, the flexible transparent micro-pillar array except the bottom and other parts of the micro-pillars except the top part are coated with a light-blocking material layer, so that light can only pass through the top of the micro-pillar and transmit to the image sensor, and the image sensor can use The photoelectric conversion principle converts the received image information into electrical signals. The left picture in the B picture in Figure 2 is a schematic diagram of the light path after the light blocking material layer is coated on the above position of the micropillar array. As shown in the figure, light-colored vertical light passes through the top of the micropillar and transmits to the image On the sensor, but other non-vertical light is isolated by the light-blocking material layer. The right picture in the B picture in Figure 2 is the final obtained micro-column deformation image.

As can be seen from Figure 2, since the light of the light-emitting film is divided into direct light and scattered light, when the light-emitting film is tightly attached to the micro-column array, in addition to the direct light transmitted to the image sensor through the micro-column from top to bottom, the scattered light It is also possible to penetrate the micropillar array from various angles, so that the image sensor cannot effectively capture the micropillar shape.

In order to clearly capture the bending and deformation of the micropillars, in the image-based tactile sensing method provided by the embodiment of the present invention, all other parts except the top of the micropillar (and the bottom of the flexible transparent micropillar array) are thinly coated to block light Materials, such as SILC pig black, can effectively block the scattered light of the luminescent film from propagating in the micro-pillar array, so that the light can only be transmitted from the top of the micro-pillar to the image sensor, so that the micro-pillars can be clearly captured. The bending and deformation of the column realize the three-dimensional measurement of the three-dimensional force. The image-based tactile sensing method provided by the embodiment of the present invention can effectively improve the clarity of the imaging, filter out the influence of the external light on the measurement result, and improve the measurement accuracy.

Optionally, an embodiment of the present invention provides an image-based tactile sensing method, further comprising adding a variable color rendering layer on the inner side of the light-blocking material layer close to the micro-pillars.

Among them, the variable color rendering layer can present different color changes according to the change of the physical quantity of the environment in which it is located. For example, different variable color developing layers can show different colors according to the different components, according to the changes of ultraviolet intensity, illumination intensity, current (or voltage) intensity and pressure intensity. The color change information of the variable color rendering layer can be obtained through the image sensor, and the measurement of environmental physical quantities within a certain range can be realized by using the image recognition algorithm.

It should be mentioned that, in order to facilitate the presentation of the contents of the embodiments of the present invention, in the following description, the variable color developing layer is a temperature-controlled color-changing layer as an example for description, which will not be repeated. Among them, the temperature-controlled color-changing layer can present different colors according to the temperature of the environment where it is located.

Alternatively, a temperature-controlled color-changing layer whose color can change with temperature can also be thinly coated on all other parts except the top of the micro-pillar.

Further, an image sensor is used to obtain the color change information of the temperature-controlled color-changing layer, and then an image recognition algorithm (such as KNN clustering algorithm, etc.) is used to measure the temperature (or other corresponding physical quantities) within a certain range.

Further, the material that changes with temperature used in the embodiment of the present invention may be a temperature-change liquid crystal (such as a temperature-change liquid crystal with a temperature detection range of 28°C-43°C, or a temperature-changeable liquid crystal with a temperature detection range of 28° C. to 43° C. can be selected according to actual use conditions. Material).

In the image-based tactile sensing method provided by the embodiment of the present invention, by adding a temperature-controlled color-changing layer at an appropriate position of the flexible transparent micro-pillar array, the image sensor is used to acquire the three-dimensional force perception data of the micro-pillar deformation image while acquiring the micro-pillar deformation image. At the same time, the color change information of the temperature-controlled color-changing layer is obtained, and the ambient temperature of the flexible transparent micro-pillar array is obtained, which realizes the integration of internal measurement and temperature measurement, and increases the miniaturization and high integration of the sensor. functional development.

Based on the content of the above-mentioned embodiments, as an option, before using the light-emitting source to project light toward the top of the micro-pillars, the method may further include: setting the light-emitting source as a light-emitting film; coating the top of each micro-pillar with a light-blocking material layer; spin-coating the light-emitting film on the light-blocking material layer.

The embodiment of the present invention provides another image-based tactile sensing method. As shown in FIG. 4 , a micro-pillar array is placed directly above the image sensor, and then the top of each micro-pillar of the flexible transparent micro-pillar array is coated with A light-blocking material layer; finally, a light-emitting film is attached to the outer layer of the light-blocking material layer to provide a light source for the flexible transparent micro-pillar array.

Since there is a light-blocking material layer between the light-emitting film and the top of the micro-pillars, the light will not pass through the top of the micro-pillars, nor can it be completely scattered, and a shadow will be cast under the micro-pillars, which will form at the bottom of the flexible transparent micro-pillar array. A shadow image. An image sensor (such as CCD or COMS) arranged at the bottom of the flexible transparent micro-pillar array is used to acquire the image information of the transmitted light, and the micro-pillar deformation after the three-dimensional force of the flexible transparent micro-pillar array is generated according to the acquired image. image.

Optionally, the micro-pillars in the above-mentioned embodiments are widened micro-pillars, and the diameter of the widened micro-pillars is 500 μm-1 cm.

After repeated experiments, it is found that, using the image-based tactile sensing method described in the above embodiment, if the micro-column is set to widen the micro-column, that is, the diameter of the micro-column is appropriately increased, a clearer projected shadow can be obtained, so that the The final generated micro-column deformation image more closely reflects the force of the micro-column, especially when the diameter of the widened micro-column is between 500μm-1cm, it can effectively improve the detection accuracy while taking into account the device integration. .

Optionally, in the image-based tactile sensing method provided by the embodiment of the present invention, on the basis of the above, a temperature-controlled color-changing layer is further added on the inner side of the light-blocking material layer close to the micro-pillars.

Specifically, in the embodiment of the present invention, between the top of the micro-pillar and the light-blocking material layer, a thin layer of a material whose color can change with temperature (or a material whose color can change with other physical quantities, such as ultraviolet, light, electricity, and pressure) is thinly coated. etc.), and obtain the color change information through the image sensor, and finally use the image recognition algorithm to measure the temperature within a certain range.

Based on the content of the above embodiment, as an option, before using the light-emitting source to project light toward the top of the micro-columns, the method further includes: setting the light-emitting source as a fluorescent coating; coating the top of each micro-column with a fluorescent coating ; Coating a light-blocking material layer on other parts of the flexible transparent micro-pillar array except the bottom.

FIG. 5 is a schematic diagram of another image-based miniaturized tactile sensing device and its working principle according to an embodiment of the present invention. As shown in FIG. 5 , the pressure sensing device involved in this embodiment is arranged above the image sensor. An array of shaped transparent micropillars, with a fluorescent coating spin-coated on top of each micropillar to provide a light source. Then, all other parts of the micro-pillar array except the bottom are coated with light-blocking material to isolate external light, so that the light will pass through the top of the micro-pillars and be transmitted to the image sensor, and finally convert the image information into electrical information through photoelectric conversion. , and reconstruct the micro-column deformation image according to the electrical information.

Alternatively, in the embodiment of the present invention, the temperature measurement method is appropriately adjusted, and the color can be changed with the physical quantity of the environment by coating a thin layer on the entire upper surface of the micro-pillar array (except the bottom of the flexible transparent micro-pillar array). The color change information can be obtained through the image sensor, and the physical quantity color image can be obtained. Finally, the image recognition algorithm is used to identify the color image of the physical quantity, so as to realize the measurement of the physical quantity within a certain range.

Based on the content of the foregoing embodiment, as an optional embodiment, before using the light source to project light toward the top of the micro-columns, the method may further include: coating the top of each micro-column with a light-blocking material layer; The outer layer of the material layer is spin-coated with a light-emitting source; a grating layer is arranged between the light-emitting source and the light-blocking material layer, and the grating layer is used to make the light incident vertically to the top of the micro-column; the light-emitting source is set as a light-emitting film or a fluorescent coating Floor.

Specifically, FIG. 6 is a schematic diagram of another image-based miniaturized tactile sensing device and its working principle according to an embodiment of the present invention. As shown in FIG. 6 , by arranging a transparent flexible micro-pillar array above the image sensor, the transparent The top of each micro-column of the flexible micro-column array is coated with a light-blocking material layer, and a light-emitting film is further attached to the top surface of each micro-column (ie, the outer layer of the light-blocking material layer) to provide a light source. Finally, in order to filter out scattered light and ensure that the light passing through the top becomes vertical light from top to bottom, a grating layer is added between the micro-pillar array and the light-emitting film. With the above arrangement, the light emitted by the light-emitting source will not pass through the top of the micro-column. Since the light is vertical after passing through the grating layer and will not be scattered, a shadow image will be cast under the micro-column. The shadow image is captured by the image sensor, and the deformation image of the micro-column is generated according to the shadow image under each micro-column. at last. The decoupling network model is used to analyze and process the deformation images of the micropillars, and the 3D perception data of the 3D force exerted on the flexible transparent micropillar array is obtained.

Alternatively, a temperature-controlled color-changing layer whose color can change with temperature may be thinly coated on the top of each micro-pillar in the pressure sensing device involved in the embodiment of the present invention, and the temperature-controlled color-changing layer is disposed on the light-blocking material. The layer is close to the inside of the micropillars.

An embodiment of the present invention provides an image-based miniaturized tactile sensing device, as shown in FIG. 7 , which mainly includes: a light-emitting source 11 , a flexible transparent micro-pillar array 21 , an image sensor 31 and an image processing unit 41 ;

The light-emitting source 11 is mainly used to project light toward the top of the micro-pillars, so that the light is emitted through the micro-pillars and the substrate; the micro-pillars are located on the substrate and constitute a flexible transparent micro-pillar array 21 ;

The image sensor 31 is placed at the bottom of the flexible transparent micro-pillar array to receive the emitted light and generate a micro-pillar deformation image; the image processing unit 31 pre-stores a decoupling network model for inputting the micro-pillar deformation image to the decoupling network model , output the three-dimensional perception data of the three-dimensional force exerted on the flexible transparent micro-pillar array; wherein, the decoupling network model is based on the sample spot image of the flexible transparent micro-pillar array 21 subjected to standard pressure and the corresponding identification label for training obtained later.

Specifically, the flexible transparent micro-pillar array may include a plurality of flexible transparent micro-pillars, and all the flexible transparent micro-pillars are arranged in an array according to actual application requirements to form the flexible transparent micro-pillar array. The arrangement is specifically limited.

The shape of the micro-pillars is not limited to circular micro-pillars, triangular micro-pillars, polygonal micro-pillars, etc.; the diameter of the micro-pillars can be set from 1 μm to several cm, etc.; the height of the micro-pillars can be adjusted according to the diameter of the micro-pillars, generally 0.1 times to 10 times the height of the pillars, and the spacing of the micropillars is more than 0.1 times the height of the micropillars.

The light-emitting source may be a flexible film that can emit light, such as a luminous film, an LED film, a fluorescent coating, etc., which is not specifically limited in this embodiment.

The image-based miniaturized tactile sensing device provided by the embodiment of the present invention uses the pre-trained decoupling network model according to the obtained micro-pillar deformation image generated by the flexible transparent micro-pillar array under the action of the three-dimensional force exerted on it. It is designed based on one of the principles of analyzing the micro-column deformation image to obtain the three-dimensional perception data of three-dimensional force.

Because the acquisition of micro-column deformation images is affected by many factors that interfere with the external environment, especially in the working environment with low light, images that can clearly reflect the micro-column deformation are often not obtained, or even cannot work normally at all. Therefore, in the image-based miniaturized tactile sensing device provided by the embodiment of the present invention, a light source is added, and the light source is used to project light from a direction perpendicular to the plane where the top of the micro-pillar is located. If there is no occlusion on the entire optical path from the top of the micro-pillar to the substrate, the light will form a complete bright spot image at the bottom of the flexible transparent micro-pillar array; there is partial occlusion on the entire optical path from the top of the micro-pillar to the substrate In the case of , a incomplete bright spot image is formed at the bottom of the flexible transparent micro-pillar array; if the entire light path from the top of the micro-pillar to the substrate is completely blocked, a shadow image is formed at the bottom of the flexible transparent micro-pillar array.

Further, an image sensor (such as CCD or CMOS, etc.) disposed at the bottom of the flexible transparent micro-pillar array is used to acquire the image information of the transmitted light, and according to the acquired image, the three-dimensional force of the flexible transparent micro-pillar array is generated. Image of micropillar deformation.

Finally, the micro-column deformation image is input into the pre-trained decoupling network model to decouple the 3D force, and the 3D perception data of the 3D force is obtained.

The image-based miniaturized tactile sensing device provided by the embodiment of the present invention can clearly capture the bending and deformation of the micro-pillar by integrating the light source in the pressure sensor, and use an intelligent algorithm based on the deformation image to complete the detection of the three-dimensional force The decoupling of the sensor realizes the miniaturization, high integration and precision of the sensor.

Based on the contents of the foregoing embodiments, optionally, the image-based miniaturized tactile sensing device provided by the embodiments of the present invention further includes a light-blocking material layer and a temperature-controlled color-changing layer; the light-emitting source is a light-emitting film, which is spin-coated on the micro- the top of the column; the light-blocking material layer is arranged on the flexible transparent micro-column array except the bottom and other parts of the micro-column except the top part; the temperature-controlled color changing layer is located on the inner side of the light-blocking material layer close to the micro-column.

Further, an embodiment of the present invention provides a method for preparing the above-mentioned image-based miniaturized tactile sensing device, as shown in FIG. 8 , including but not limited to the following steps:

Q1, prepare flexible transparent micro-pillar array;

Q2, coating a variable color developing layer on the flexible transparent micro-pillar array;

Q3, coating light blocking material on the variable color developing layer.

Wherein, the preparation of flexible transparent micro-pillar array described in step Q1 mainly includes:

The photoresist is spin-coated on the inner surface of the preparation container; the preparation container is exposed and developed; the preset pattern of the flexible transparent micro-pillar array is set on the photoresist on the inner surface of the preparation container to make flexible transparent micro-columns. The mold of the column array; after the prepolymer for making the flexible transparent micro-pillars in the whole array of flexible transparent micro-pillars and the silicone oil are uniformly mixed according to the first set ratio, poured on the mold, and after heating and curing, the flexible transparent micro-pillars are molded into flexible Transparent micropillar array.

Wherein, coating the variable color developing layer on the flexible transparent micro-column array described in step Q2 mainly includes: bonding a glass sheet on the top of the micro-column; Mix uniformly in a fixed proportion to generate a mixed liquid of color developing material; after injecting the mixed liquid of color developing material into the gap between the micro-pillars of the flexible transparent micro-pillar array, blow out the excess mixed liquid of color developing material with gas; heat the mixed liquid of color developing material to mix the color developing material After the liquid has solidified, remove the glass slide.

Wherein, in step Q3, coating the light-blocking material layer on the variable color developing layer mainly includes: re-bonding the glass sheet on the top of the micro-column; uniformly mixing the liquid light-blocking material and the prepolymer at a third set ratio , generate the light-blocking material mixed liquid; after injecting the light-blocking material mixed liquid into the gap between the micro-pillars of the flexible transparent micro-pillar array, blow out the excess light-blocking material mixed liquid with gas; after heating the light-blocking material mixed liquid to solidify, Remove the glass sheet.

FIG. 9 is a process flow diagram of preparing a flexible transparent micro-pillar array in an image-based miniaturized tactile sensing device provided by an embodiment of the present invention. As shown in FIG. 9 , the steps of preparing the transparent flexible micro-pillar array may be specifically:

(1) Put the PDMS prepolymer (liquid) and the silicone oil into the beaker according to the required ratio, and stir it for 2 hours with a planetary stirrer to prepare a PDMS prepolymer and silicone oil composite (liquid) for use.

(2) Cleaning the wafer and performing plasma cleaning, spin-coating SU-8 on the silicon wafer, exposing and developing, transferring the designed pattern to SU-8 to make a mold.

(3) The composite of PDMS prepolymer and silicone oil and the curing agent are mixed uniformly in a ratio of 10:1, and the air bubbles are removed under vacuum, and then poured on the SU8 template.

(4) After heating and curing, the mold is inverted to form a micro-channel structure with micro-pillars.

Further, the preparation process of the variable color-developing layer (temperature-controlled color-changing layer) can be specifically:

(1) Bond the top of the micro-pillar of the sensor sheet to the glass sheet.

(2) The temperature-changeable liquid crystal and PDMS are mixed in a ratio of 2:1, and the mixture is injected into the gap between the micro-columns with a syringe, and the excess liquid is driven out by air blowing.

(3) heating to solidify the temperature-change liquid crystal material.

(4) After the temperature-changeable liquid crystal material is solidified, the bonded glass sheets are removed to obtain a sensor sheet with only the tops of the micro-pillars not coated with temperature-changeable liquid crystals.

Further, the preparation process of the light blocking material layer can be specifically as follows:

(1) Bond the top of the micro-pillar of the sensor sheet to the glass sheet.

(2) Mix SILC pig black and PDMS at a ratio of 2:1, inject the mixture into the gap between the micro-columns with a syringe, and blow the excess liquid out.

(3) Heating to solidify the black dye.

(4) After the black pigment is solidified, the bonded glass sheet is removed to obtain a sensor sheet with only the top of the micro-pillar not painted black.

The image-based miniaturized tactile sensing device prepared by using the process flow provided by the embodiment of the present invention does not require electronic components such as capacitors and resistors, so the volume of the measuring unit and the number of sensor connections can be effectively reduced, and the It can achieve the miniaturization and high integration of the sensor. At the same time, because it collects image information instead of electrical signals, it is not easily affected by external environments such as temperature or electromagnetic interference. At the same time, due to the use of arrayed flexible transparent micro-pillars as the sensitive element, and the light source inside the sensor element, the detection sensitivity of horizontal force (lateral force) is high, the direction of the three-dimensional force can be accurately identified, and it is affected by the outside world. smaller. Further, the three-dimensional force measurement device provided by the embodiment of the present invention can be attached to substrates with different surface topography like human skin, and can also acquire image information accurately and quickly, which is very useful for the realization of artificial touch. has far-reaching significance. There are extensive demands for flexible 3D force sensors in the fields of intelligent robots, mechanical assembly, automobile manufacturing, and medical treatment.

As will be appreciated by one skilled in the art, embodiments of the present invention may be provided as a method, system, or computer program product. Accordingly, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) having computer-usable program code embodied therein.

The present invention is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each flow and/or block in the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to the processor of a general purpose computer, special purpose computer, embedded processor or other programmable data processing device to produce a machine such that the instructions executed by the processor of the computer or other programmable data processing device produce Means for implementing the functions specified in a flow or flow of a flowchart and/or a block or blocks of a block diagram.

It should be noted that the terms "comprising", "comprising" or any other variation thereof are intended to cover a non-exclusive inclusion such that a process, method, article or device comprising a series of elements includes not only those elements, but also includes no explicit Other elements listed, or those inherent to such a process, method, article or apparatus. Without further limitation, an element qualified by the phrase "comprising a..." does not preclude the presence of additional identical elements in a process, method, article or apparatus that includes the element.

In the description of the present invention, numerous specific details are set forth. It will be understood, however, that embodiments of the invention may be practiced without these specific details. In some instances, well-known methods, structures and techniques have not been shown in detail in order not to obscure an understanding of this description. Similarly, it is to be understood that in the above description of exemplary embodiments of the invention, various features of the invention are sometimes grouped together into a single embodiment in order to simplify the present disclosure and to aid in the understanding of one or more of the various aspects of the invention. , figures, or descriptions thereof. However, this method of disclosure should not be construed to reflect the intention that the invention as claimed requires more features than are expressly recited in each claim. Rather, as the following claims reflect, inventive aspects lie in less than all features of a single foregoing disclosed embodiment. Thus, the claims following the Detailed Description are hereby expressly incorporated into this Detailed Description, with each claim standing on its own as a separate embodiment of this invention.

The above embodiments are only used to illustrate the technical solutions of the present invention, but not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: The recorded technical solutions are modified, or some technical features thereof are equivalently replaced; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions in the embodiments of the present invention.

Claims (10)

1. An image-based haptic sensing method, comprising:

projecting light rays to the top direction of the microcolumns by using a light emitting source, wherein the light rays are emitted through the microcolumns and the substrate; the micro-columns are positioned on the substrate to form a flexible transparent micro-column array;

receiving the emitted light by using an image sensor arranged at the bottom of the flexible transparent micro-column array to generate a micro-column deformation image;

inputting the micro-column deformation image into a decoupling network model, and outputting three-dimensional sensing data of three-dimensional force applied to the flexible transparent micro-column array;

the decoupling network model is obtained after training based on a sample light spot image of the flexible transparent micro-column array subjected to standard three-dimensional force and a corresponding identification label.

2. The method of claim 1, further comprising, before the projecting light to the direction of the tips of the micro-pillars by the light emitting source:

arranging the light-emitting source as a light-emitting film;

spin-coating the light-emitting thin film on the top end of the micro-column;

coating the light blocking material layer on the bottom of the flexible transparent micro-column array and other parts of the micro-columns except the top parts.

3. The method of claim 1, further comprising, before the projecting light to the direction of the tips of the micro-pillars by the light emitting source:

arranging the light-emitting source as a light-emitting film;

coating a light blocking material layer on the top end of each micro-column;

and spin-coating the light-emitting film on the light blocking material layer.

4. The image-based haptic sensing method of claim 3,

the diameter of the microcolumn is 500-1 cm.

5. The method of claim 1, further comprising, before the projecting light to the direction of the tips of the micro-pillars by the light emitting source:

providing the light-emitting source as a fluorescent coating;

coating the fluorescent coating on the top end of each micro-column;

coating a light blocking material layer on the other parts of the flexible transparent micro-column array except the bottom.

6. The method of claim 1, further comprising, before the projecting light to the direction of the tips of the micro-pillars by the light emitting source:

coating a light blocking material layer on the top end of each micro-column;

spin coating the light-emitting source on the outer layer of the light-blocking material layer;

arranging a grating layer between the light-emitting source and the light blocking material layer, wherein the grating layer is used for enabling the light to vertically enter the top end of the micro-column;

the light-emitting source is provided as a light-emitting film or a fluorescent coating.

7. The image-based haptic sensing method of any of claims 2-6, further comprising:

and a variable color development layer is additionally arranged on the inner side of the light blocking material layer close to the microcolumns.

8. An image-based miniaturized tactile sensing device, comprising:

the device comprises a light emitting source, a flexible transparent micro-column array, an image sensor and an image processing unit;

the light source is used for projecting light rays to the top direction of the microcolumn, and the light rays are emitted out through the microcolumn and the substrate;

the micro-pillars are positioned on the substrate and integrally formed to form a flexible transparent micro-pillar array;

the image sensor is arranged at the bottom of the flexible transparent micro-column array and used for receiving the emitted light and generating a micro-column deformation image;

the image processing unit is prestored with a decoupling network model and is used for inputting the micro-column deformation image into the decoupling network model and outputting three-dimensional sensing data of three-dimensional force applied to the flexible transparent micro-column array;

the decoupling network model is obtained after training based on a sample light spot image of the flexible transparent micro-column array under the action of standard pressure and a corresponding identification label.

9. The image-based miniaturized tactile sensing device according to claim 8, further comprising a light blocking material layer and a variable color development layer;

the light-emitting source is a light-emitting thin film and is spin-coated on the top end of the microcolumn;

the light blocking material layer is arranged at the bottom of the flexible transparent micro-column array and other parts of the micro-columns except the top parts;

the variable color development layer is positioned on the inner side of the light blocking material layer close to the microcolumns.

10. A method of making an image-based miniaturized touch-sensing device according to claim 9, comprising sequentially performing the steps of:

preparing a flexible transparent micro-column array, coating a variable color development layer on the flexible transparent micro-column array, and coating a light blocking material layer on the variable color development layer;

wherein, the preparation of the flexible transparent micro-column array comprises the following steps:

spin coating photoresist on the inner surface of the preparation container;

exposing and developing the preparation container;

arranging the preset pattern of the flexible transparent micro-column array on the photoresist on the inner surface of the preparation container to manufacture a mould of the flexible transparent micro-column array;

uniformly mixing prepolymer for manufacturing the flexible transparent micro-pillars in the flexible transparent micro-pillar array with silicone oil according to a first set proportion, pouring the mixture on the mold, heating and curing the mixture, and performing reverse molding to obtain the flexible transparent micro-pillar array;

the coating of the variable color development layer on the flexible transparent micro-pillar array comprises:

bonding a glass sheet on the top end of the micro-column;

uniformly mixing the liquid variable color development material and the prepolymer in a second set proportion to generate color development material mixed liquid;

injecting the color development material mixed liquid into gaps among the micro-pillars of the flexible transparent micro-pillar array, and blowing out redundant color development material mixed liquid by using gas;

heating to solidify the color developing material mixed liquid, and then taking down the glass sheet;

the coating of the light-blocking material layer on the variable color development layer comprises:

bonding a glass sheet on the top end of the micro-column again;

uniformly mixing the liquid light blocking material and the prepolymer according to a third set proportion to generate light blocking material mixed liquid;

injecting the light blocking material mixed liquid into gaps among the micro-pillars of the flexible transparent micro-pillar array, and blowing out redundant light blocking material mixed liquid by using gas;

and heating to solidify the light blocking material mixed liquid, and then taking down the glass sheet.

Images