(PDF) Tactile sensing for dexterous in-hand manipulation in robotics—A review

As the field of robotics is expanding from the fixed environment of a production line to complex human environments, robots are required to perform increasingly human-like manipulation tasks, moving the state-of-the-art in robotics from grasping to advanced in-hand manipulation tasks such as regrasping, rotation and translation. To achieve advanced in-hand manipulation tasks, robotic hands are required to be equipped with distributed tactile sensing that can continuously provide information about the magnitude and direction of forces at all contact points between them and the objects they are interacting with. This paper reviews the state-of-the-art in force and tactile sensing technologies that can be suitable within the specific context of dexterous in-hand manipulation. In previous reviews of tactile sensing for robotic manipulation, the specific functional and technical requirements of dexterous in-hand manipulation, as compared to grasping, are in general not taken into account. This paper provides a review of models describing human hand activity and movements, and a set of functional and technical specifications for in-hand manipulation is defined. The paper proceeds to review the current state-of-the-art tactile sensor solutions that fulfil or can fulfil these criteria. An analytical comparison of the reviewed solutions is presented, and the advantages and disadvantages of different sensing technologies are compared.

Sensors and Actuators A 167 (2011) 171–187 Contents lists available at ScienceDirect Sensors and Actuators A: Physical journal homepage: www.elsevier.com/locate/sna Review Tactile sensing for dexterous in-hand manipulation in robotics—A review Hanna Yousef a,∗ , Mehdi Boukallel a , Kaspar Althoefer b a b CEA, LIST, Sensory and Ambient Interfaces Laboratory, 18 route du Panorama, BP6, Fontenay-aux-Roses F-92265, France King’s College London, Department of Informatics, Strand, London WC2R 2LS, United Kingdom a r t i c l e i n f o Article history: Received 5 October 2010 Received in revised form 21 February 2011 Accepted 21 February 2011 Available online 2 March 2011 Keywords: Tactile sensing Extrinsic sensing Sense of touch Robotic skin Humanoid robots Dexterous in-hand manipulation a b s t r a c t As the field of robotics is expanding from the fixed environment of a production line to complex human environments, robots are required to perform increasingly human-like manipulation tasks, moving the state-of-the-art in robotics from grasping to advanced in-hand manipulation tasks such as regrasping, rotation and translation. To achieve advanced in-hand manipulation tasks, robotic hands are required to be equipped with distributed tactile sensing that can continuously provide information about the magnitude and direction of forces at all contact points between them and the objects they are interacting with. This paper reviews the state-of-the-art in force and tactile sensing technologies that can be suitable within the specific context of dexterous in-hand manipulation. In previous reviews of tactile sensing for robotic manipulation, the specific functional and technical requirements of dexterous in-hand manipulation, as compared to grasping, are in general not taken into account. This paper provides a review of models describing human hand activity and movements, and a set of functional and technical specifications for in-hand manipulation is defined. The paper proceeds to review the current state-of-the-art tactile sensor solutions that fulfil or can fulfil these criteria. An analytical comparison of the reviewed solutions is presented, and the advantages and disadvantages of different sensing technologies are compared. © 2011 Elsevier B.V. All rights reserved. Contents 1. 2. 3. 4. 5. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Human in-hand manipulation as a basis for robotic manipulation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.1. Human hand and finger movements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.2. Tactile sensing in the human hand . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.3. Specification for tactile sensing for in-hand manipulation for robotics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Technologies for tactile sensing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.1. Resistive sensors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.1.1. Micromachined strain gauges . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.1.2. Micromachined piezoresistors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.1.3. Conductive polymers and fabrics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.1.4. Conductive elastomer composites . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.1.5. Conductive fluids . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.2. Capacitive sensors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.3. Piezoelectric sensors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.4. Optical sensors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.5. Organic field-effect transistors (OFETs) as sensors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A comparison of sensor solutions and sensing techniques . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Summary and conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Biographies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ∗ Corresponding author. E-mail address: hanna.yousef@cea.fr (H. Yousef). 0924-4247/$ – see front matter © 2011 Elsevier B.V. All rights reserved. doi:10.1016/j.sna.2011.02.038 172 172 172 173 173 174 174 174 178 179 180 181 181 182 183 184 184 185 185 185 187 172 H. Yousef et al. / Sensors and Actuators A 167 (2011) 171–187 1. Introduction “For robots, the final frontier is not space; it is your living room” [1]. The field of robotics today is continuously expanding from the fixed environment of a production line to include more complex environments such as homes, offices, and hospitals. The new application areas require versatile autonomous intelligent robots that can interact with humans and their wide range of tools in real-world environments. To perform increasingly human-like functions, robots are required to be able to perform increasingly human-like manipulation tasks, moving the state-of-the-art in robotics from grasping to advanced manipulation tasks such as in-hand regrasping, rotation and translation. To intelligently perform in unstructured and changing surroundings, robots will be required to manipulate objects while simultaneously sensing and reasoning about their environment. To achieve this, robots need an interface that can provide information about the forces and positions at all points of contact between them and the objects they are interacting with. A key issue in the robotics community today is therefore the development of artificial skin interfaces with fully distributed tactile sensing. Tactile sensing in robotics is defined as the continuous sensing of variable contact forces [2]. This information can be used to determine if the robot is in contact with an object, the contact configuration, the stability of the grasp, as well as for force feedback for the control of the robot [3]. Furthermore, tactile information is envisaged to be used to analyse object manipulation to better understand and optimise handling techniques so as to further increase the versatility, skills and performance of the robot [4]. A thorough review of the state of the art in tactile sensing for mechatronics in general is presented by Lee and Nicholls [5]. Different technologies and application areas are reviewed including sensing fingers, industrial grippers and multifingered hands for dexterous manipulation. A more recent review by Saraf and Maheshwari [6] includes an outlook on potential high-performance devices based on recent research in nanostructured materials. In addition to these general reviews, several articles reviewing sensors for specific applications areas are presented such as for ‘smart skins’ [7], minimally invasive surgery [8], robotics in medicine, prosthetics and the food industry [9], and for robotic dexterous manipulation in [3,10]. In the aforementioned articles, although sensor specifications are discussed for robotics, the functional and technical requirements of dexterous in-hand manipulation, as compared to grasping, are in large not taken into account. This paper provides a review of the current state-of-the-art in tactile force and pressure sensing within the specific context of dexterous in-hand manipulation. Taking human in-hand manipulation as a basis for understanding the specific requirements for in-hand manipulation, a review of models describing human hand activity and movements is presented and a set of functional and technical specifications on a robotic tactile sensor system is defined. The literature reviewed deals with sensors that fulfil these criteria, as well as sensors that in our opinions can be adapted to fulfil them. An analytical comparison of the reviewed work is presented and the advantages and disadvantages of different sensing technologies are compared. 2. Human in-hand manipulation as a basis for robotic manipulation As robots are required to perform increasingly human-like manipulation in unstructured environments, the tendency in the robotics community is to look to human movements, as well as the human skin and sense of touch, for inspiration. It is therefore of interest to understand the physiology of the human sense of touch and perception, as well as the ergonomics of human hand activity and movements during grasping and in-hand manipulation of objects. The former has been treated in the robotics community, e.g. as reviewed in [10]. An understanding of the latter in the context of robotics we find is however still lacking. 2.1. Human hand and finger movements Human in-hand object manipulation consists of a series of actions, each fulfilling a sub-task of the manipulation task. Personal constraints aside, the chosen actions to perform a manipulation task depend on object related parameters such as size, weight, shape and texture, manipulation related parameters such as movement patterns, and performance demands such as speed and accuracy [11]. Hand postures and movements for grasping objects have been widely studied, and a large amount of work on modelling and replication can be found, e.g. [12–17]. In comparison, in-hand manipulation has not been studied to the same extent. This can be attributed to the high complexity and diversity of the tasks, as well as to the limitations of available sensing technologies with regard to sensitivity and spatial resolution. In-hand manipulation has however been studied within the fields of medicine, developmental psychology, sensory integration therapy and physical therapy [18–22]. Two main systems for classification of hand movements for in-hand manipulation can be found [23,24]. Elliot and Connolly classify in-hand manipulation with regard to the movements of the fingers involved in the manipulation [23]. Here three main classes are identified: (1) simple synergies when all the participating digits move as one unit, bending or extending, e.g. when squeezing a small ball or pipette, (2) reciprocal synergies when the thumb moves independently while the remaining involved digits move as one, e.g. when screwing/unscrewing the lid of a bottle, and (3) sequential patterns when the participating fingers move independently of each other to form movement patterns, e.g. during turning and/or repositioning of a pen in the hand. In addition to the movement of the fingers, the authors introduce a class of movements, palmar combinations, where the manipulated object is immobilised by the palm of the hand while the participating digits manipulate another part of the object, e.g. when screwing/unscrewing the lid of a tube while holding with the same hand. In Exner’s classification system [24], the amount and type of displacement of the object in the hand is taken into account in addition to the movement of the hand. Here, three main categories are identified: (1) translation when an object is moved from the fingertips to the palm of the hand, or from the palm to the fingertips, e.g. picking up multiple small object and storing in the hand, (2) shift when the object is moved linearly along or across one or more fingertips, e.g. when repositioning a pencil for writing, and (3) rotation when an object is turned around in the pads of the fingers and thumb (simple) or when rolling an object or turned from end to end (complex), e.g. when flipping a pen around to reposition for writing. Pont et al. [25] further develop Exner’s classification system to include the complexity of the finger motion required to achieve the manipulation, as well as including a specific focus on the need for stabilisation. In this way, Pont et al. present a system that is consistent with both Exner as well as with Elliot and Connolly. In this system, Exner’s “shift” is further divided into simple and complex shifts. Here, simple shifts combine Exner’s shift with Elliot and Connolly’s simple synergies, and complex shifts combine shift with sequential patterns. Furthermore, the authors discuss that the importance of translation from fingers to palm is mainly to achieve stability. In the different movements described in the three systems above, it can be seen that the pads of all five fingers at the dis- H. Yousef et al. / Sensors and Actuators A 167 (2011) 171–187 173 Fig. 1. Density of mechanoreceptors (afferents per cm2 ) in the hand. (a) Fast adapting type I, (b) slow adapting type I, (c) fast adapting type II and (d) slow adapting type II. Colour coding for all four figures is shown in (a). Reprinted by permission from Macmillan Publishers Ltd: Nature Reviews Neuroscience [26], Copyright 2009. tal phalanges are involved in direct manipulation of objects in a large number of in-hand manipulation tasks. The fingertips are also involved in maintaining grasp stability by the application of forces normal to the object surface to counter-act tangential forces that arise due to slip, rotation of the object and its weight [26]. Furthermore, the normal grasp forces are varied to compensate for varying object shape, surface friction, inertia, elasticity and viscosity [27]. It can hence be deduced that grasp stabilisation is important to prevent slip as well as to allow for transitions between force and precision grips. The finger pads at the intermediate and distal phalanges can however be seen as mostly necessary for stabilising the object. Similarly, the sides of the five fingers and the palm of the hand are mostly used for stabilisation of the object. In addition during in-hand manipulation, the involvement of multiple digits leads to synergetic effects between the forces applied by each finger. In fact, for a five finger grasping configuration the manipulated object undergoes more than 30 mechanical interactions of (3 forces, 3 moments for one finger). During different manipulation tasks, the same finger may act as a slave finger or a master finger. This can lead to redundant mechanical interactions on the object [28,29]. Santello et al. [30] emphasise the link between grasping configuration and contact forces. The authors demonstrate that in static hand posture of the fingers and the thumb (modelled with more than 15 joint angles), the joint angles of the digits (fingers) do not vary independently. Experiments conducted on multi-shaped objects show that hand posture can be controlled independently from the contact forces needed to grasp objects. The results underline the control complexity due to the multi fingering configuration. The above factors lead to increased complexity in finger control and coordination even for simple shaped objects, highlighting the need for a better understanding of the mechanisms of in-hand manipulation as compared to grasping. in [26,34,35]. In summary, four different types of afferents have been identified, each with their function and sensing range. The mechanoreceptors are characterised with regard to their response speed and hence the stimuli they respond to. Two types of fast adapting afferents (type I and type II) respond to temporal changes in skin deformations (dynamic). Two types of slow adapting afferents (type I and type II) respond to sustained deformations over time (static). The mechanoreceptors are further categorised with regard to their location in the depth of the skin and hence their receptive field, i.e., the area of the outer skin in which the afferent responds when stimulated. The type I afferents are located in the dermal-epidermal boundary and have small and well-defined receptive fields, while the type II afferents are found in deeper layers of the skin and have larger and more diffuse receptive fields. The density of the type I afferents is highest at the fingertips and decreases proximally, while type II afferents are more uniformly distributed throughout the fingers and palm of the hand (see Fig. 1). Furthermore, there is a predominance of fast adapting type I afferents in the hand. These two points indicate the high significance of high spatial and temporal resolution in dynamic mechanical interactions, typically during the making, breaking or variation of contact. This supports the suggestion in the section above that the fingertips and distal phalanges are mainly responsible for movements for direct manipulation of objects, and that tactile sensory signals from the entire hand, albeit at lower temporal and spatial resolution, are critical for maintaining stability during manipulation. 2.3. Specification for tactile sensing for in-hand manipulation for robotics 2.2. Tactile sensing in the human hand Based on the discussion above, the minimum functional requirements for a robotic tactile sensing system mimicking human in-hand manipulation can be summarised in the following points. Each of the movements described above is characterised by distinct mechanical contact events as for example the making or breaking of contact. Consequently, each sub-task generates distinct and discrete sensory signals [31]. Of the sensory modes involved (mainly tactile, proprioceptive, and visual), tactile sensing provides a direct measurement of mechanical contact events and interactions [32]. Tactile signals are hence critical control points for the start, duration and end of each interaction, as well as the adaptation of predicted and applied forces to the object and manipulation at hand. Tactile sensory signals due to contact events are provided by mechanoreceptive afferent neurons (mechanoreceptors) that innervate the outer layers of the skin [33]. The different types of mechanoreceptors, their density and characteristics, as well as the coding and function of the generated signals are reviewed • Detect the contact and release of an object. • Detect lift and replacement of an object. • Detect shape and force distribution of a contact region for object recognition. • Detect contact force magnitude and direction for maintaining a stable grasp during manipulation. • Detect both dynamic and static contact forces. • Track variation of contact points during manipulation. • Detect difference between predicted and actual grip forces necessary for manipulation. • Detect force and magnitude of contact forces due to the motion of the hand during manipulation. • Detect tangential forces due to the weight and shape of the object to prevent slip. 174 H. Yousef et al. / Sensors and Actuators A 167 (2011) 171–187 Table 1 Design guidelines for tactile sensing system for in-hand manipulation. Parameter Guidelines Force direction Temporal variation Spatial resolution (point to point)a Both normal and tangential Both dynamic and static 1 mm at fingertips to 5 mm in palm of hand 1 msb 0.01–10 N (1000:1) Stable, repeatable, and monotonic. Low hysteresis Withstand application defined environment Minimal cross-talk Electronic and/or magnetic shielding Time responseb Force sensitivity (dynamic range) Linearity/hysteresis Robustness Tactile cross-talk Shielding Integration and fabrication Simple mechanical integration Minimal wiring Low power consumption and cost Adapted from [10,34,37]. a Two-point spatial resolution is defined as the smallest distance between two simultaneous point contacts which can be resolved. b For a single sensor element in an array. • Detect tangential forces arising from variations in object parameters (e.g. surface friction, elasticity, etc.) to prevent slip. Dargahi and Najarian present general design guidelines based on mimicking human tactile sensing while considering the limitations and possibilities of, e.g. measurement and data processing units that are coupled to the sensor system [34]. The guidelines are based on mimicking tactile sensing at the fingertips where the concentration of mechanoreceptors is at its highest. Dahiya et al. add several considerations to the suggested design guidelines, taking into consideration the need for different types of sensors as is found in the human skin [10]. Furthermore, they argue that some processing of tactile data can be done locally before sending to the central processing unit so as to reduce the amount of information. This point is further discussed and reviewed in [36]. It is also suggested that different types of tactile data can be transferred via different paths at different rates, analogous to the fast and slow adapting mechanoreceptors, so that more ‘urgent’ signals can be treated quicker. However, this may lead to an undesirable increase in wiring. A set of design guidelines adapted to in-hand manipulation, is shown in Table 1. To achieve the design guidelines mentioned above, a tactile sensing system can either consist of sensors that fulfil the aforementioned criteria, or alternatively hybrid solutions combining different sensors that collectively fulfil the criteria. An envisaged solution is to integrate highly sensitive miniaturised 3D force sensors on larger-area pressure sensitive substrates with lower spatial resolution. 3. Technologies for tactile sensing The reviewed sensor solutions are presented with regard to their transduction method. A comparison of individual sensor solutions is found in Section 4 (Table 2 ). A general comparison of the advantages and disadvantages of the different sensing techniques is found in Section 4 (Table 3). 3.1. Resistive sensors 3.1.1. Micromachined strain gauges Strain gauges consist of a structure that elastically deforms when subjected to a force which in turn leads to a change in its resistance. To optimise the change in resistance due to applied mechanical stress, strain gauges are typically long winding snakelike structures. In this way, when deformed, the cross-section of the strain gauge decreases while its conduction length increases. Here, typically, the change in resistance of the strain gauge material itself is secondary to the change due to its mechanical deformation. Micromachined strain gauges have the advantage of high sensitivity, small sizes, high spatial resolution, and well-established fabrication techniques. Furthermore, the strain gauges can be directly integrated with readout electronics and other microelectromechanical systems (MEMS) elements. Xu et al. present flexible strain gauge sensor skins especially developed for curved surfaces [38,39]. Here, silicon-based IC strain gauges are bonded onto a flexible printed circuit board (flex PCB) (see Fig. 2a). Each sensor package has an area of 10 mm × 20 mm and consists of a 1-D array of 16 sensors. The signal processing circuitry is included in the same package increasing reliability and robustness. In [40,41] islands of diffused silicon strain gauges are directly encapsulated in parylene or polyimide, forming a highly flexible network (see Fig. 2b and c). Furthermore, stitching holes are incorporated into the flexible sensor skin to allow for integration with textiles. A 4 × 4 sensor array is presented in an area of 22 mm × 21 mm. Preliminary tests show that the sensor network can withstand stretching and twisting, however with non-linear behaviour. Metallic strain gauges on flexible polyimide films are demonstrated by several groups [42–46]. Metals are chosen as the strain gauge material as they are often deposited using temperatures that are compatible with polyimide. The strain gauges are often placed at points of maximum stress of a diaphragm in the flexible film, and bumps are often added on top of the diaphragm to improve sensitivity (see Fig. 3). In [42], Kim et al. present arrays of 32 × 32 nickel–chromium (NiCr) strain gauges in a polyimide layer in a total area of 55 mm × 65 mm. By applying thick layers of polyimide (80 ␮m), a higher deflection of the strain gauge is possible, and the authors achieve a higher sensitivity range than for previously presented NiCr strain gauges on polyimide [43]. However, in this case the spatial resolution and mechanical flexibility of the sensor is somewhat decreased. The sensors show a linear Fig. 2. (a) Silicon flexible skin wrapped around a half-inch diameter aluminium block. (b) Schematic illustration of an approach to integrating silicon flexible skin with textiles. (c) Photograph of silicon skin stitched onto a canvas fabric. Reprinted from [41], Copyright 2005, with permission from Elsevier. Table 2 Comparison of the reviewed sensor solutions. Reference Author Sensor functionality Year Ref. 3D force sensors—arrays Kim 2006 [48] Sensing principle Force/pressure sensitivity or resolution [N]a Ratio (N/S)Force/pressure range [N]a Normal Shear Strain gauge 2.1% 0.5% Mechanical properties, size and application Array Mechanical flexibility No. of elements Spatial Total size resolution [mm2 ] [mm] Side length [mm] Area of suitable use Shear Dist. ph. Prox. ph. Palm 4.2 0–2 0–2 Embedded 1.5 x x 4×4 1.8 49 x 4×4 n.s. n.s. 3×3 1 9 x 8×8 2.75 484 x – – – 2010 [45] Strain gauge 207 mV 0.070 mV 3.0 0–0.8 0–0.8 Flexible 2.5 x Sohgawa 2009 [49] Piezoresistor 2.2 mV 0.14 mV 15.7 0–0.13 0–0.03 Embedded 1 x Lee [81] Capacitive 3.0% 2.7% (avg) 1.1 0–0.01 0–0.01 Stretchable 2 x 3D force sensors—not in arrays [51] Piezoresistor Ho 2009 85 mV 39 mV 2.2 0–0.5 Embedded 1×5 Noda 2006 [53] Piezoresistor 0.015% 0.03% 0.5 0–4 0–4 Embedded 20 – – – Noda 2009 [55] Piezoresistor 0.01% 0.1% 0.1 0.05–3 0.05–3 Embedded 20 – – – Beccai 2008 [56] Piezoresistor 100 mV 400 mV 0.3 0–6 0–8 Flexible 3 – – – Wang 2009 [63] Cond. polymer 15 mV 8.6 mV 1.7 0–0.4 0–0.4 Stretchable 10 – – – 1.5% (avg) – – 0–1 Flexible 1 x x 32 × 32 2 3575 Flexible 0.1 x x 10 × 10 0.4 16 Flexible n.s. n.s. n.s. n.s. Flexible 1 32 × 32 1.9 8100 Stretchable 9 x n.s. 10 14,400 Flexible 3 x 32 × 32 5 27,225 Stretchable 2.5 8×8 3 400 Flexible 2.54 32 × 32 2.54 6400 Choi 2008 Pressure/normal force sensors—arrays Kim 2009 [42] Strain gauge x x x – Engel 2003 [43] Strain gauge 0.6 /␮m – – n.s. – Zhang 2010 [50] Strain gauge 0.3% – – 0–7 – Yu 2009 [61] Cond. polymer 0.1%/kPa – – 0–30 kPa x x H. Yousef et al. / Sensors and Actuators A 167 (2011) 171–187 Normal – Alirezai 2009 [67] Res—EIT Image – – 0–150 kPa – Yang 2010 [68] Cond. elastomer Image – – 20–300 kPa x – Cheng 2009 [70] Cond. elastomer 300 /kPa – – 0–650 kPa x x – Someya 2004 [73] Cond. elastomer 0.2 ␮A/kPa – – 0–30 kPa x x – 175 176 Table 2 (Continued) Reference Sensor functionality Year Ref. Sensing principle Someya 2005 [75] Cond. elastomer Force/pressure sensitivity or resolution [N]a Ratio (N/S)Force/pressure range [N]a Normal Shear n.s. – Normal – Mechanical properties, size and application Array Mechanical flexibility No. of elements Spatial Total size resolution [mm2 ] [mm] 12 × 12 4 1936 n.s. 2 Fingertip 4×4 2, 6 750 n.s. 5 Fingertip 8×8 2 256 Side length [mm] Shear 0–1 Area of suitable use Dist. ph. Prox. ph. Palm x Stretchable x – Wettels 2008 [76] Cond. fluid – – 0.01–40 Embedded 2.3 Stretchable 0.25b Embedded 0.5b x x – Hasegawa2008 [86] Optical 0.023 V – – 0–0.3 x x – Chorley 2009 [101] Optical 0.05 N – – 0.05–0.5 x – Mannsfeld2010 [107] OFET 1 (0.3) ␮A/kPa – – 0–2 (2–18) kPa Flexible x x – Pressure/normal force sensors—not arrays [94] Optical Heo 2008 0.05 N – – 0–10 Stretchable – Sato 2008 [99] Optical 0.3 N – – 0.2–2 Embedded 5 mm – Manunza 2007 [104] OFET 0.8 kPa – – 0–20 kPa Flexible – n.s. not specified; Prox. ph.: proximal phalange; Dist. ph.: distal phalange. a Unless otherwise specified. b Diameter of a circular sensor. 5 x x H. Yousef et al. / Sensors and Actuators A 167 (2011) 171–187 Author H. Yousef et al. / Sensors and Actuators A 167 (2011) 171–187 177 Table 3 Comparison of the reviewed sensing techniques. Sensor type Resistive MEMS strain gauges and piezoresistors Advantages Disadvantages •High sensitivity •Small sizes and high spatial resolution •Fragile sensor element •Relatively costly materials and fabrication techniques •When integrated with flex PCB/fabric, not stretchable •Even if sensor is small, total package size can be large •Well established design and fabrication techniques •3D force sensing possible •Ease of integration with other MEMS and electronics •Ease of integration with flex PCB/fabric for flexibility Resistive Embedded MEMS strain gauges and piezoresistors •High sensitivity •Small sizes and high spatial resolution •Well established design and fabrication techniques •Elastomer is stretchable •Elastomer as protective layer •Soft material mimics human skin •Increased grasping quality •3D force sensing possible •Ease of integration with other MEMS and electronics Resistive Conductive polymer films •Mechanically flexible •Robust and chemically resistant •Large-area low-cost fabrication techniques •Loss of sensor sensitivity •Even if sensor is small, total package size can be large •Relatively costly materials and fabrication techniques •Creep •Fragile sensor element •Ambiguity (transverse inverse problem) •Not stretchable •Low sensitivity •Conduction in all directions so applications often restricted to pressure sensing/imaging •Simple structures and fabrication techniques possible •Thin films and low weights possible Resistive Conductive elastomer composites Resistive OFET sensors Capacitive Piezoelectric (PVDF) Optical •Stretchable •Soft material mimics human skin •Increased grasping quality •Simple structures and fabrication techniques possible •Can be tailored for specific measurement ranges •Minimised wiring •Simplified fabrication process •Suitable for large-area applications •Low cost per area compared to IC transistors •Ease of integration with other flexible MEMS •High sensitivity •Temperature independent •Large area applications possible •Small sizes and high spatial resolution possible •Well established design and fabrication techniques •3D force sensing possible •High sensitivities and outputs •Well suited for dynamic applications •Mechanically flexible •Thin films and low weights possible •Robust and chemically resistant •Simplified wiring •Hysteresis of composite material •Low sensing range •Restricted to pressure sensing/imaging •Low sensitivity •Low response time as compared to IC transistors •Restricted to pressure sensing/imaging •Parasitic capacitances •Sensitive to electromagnetic interference •Relatively complex circuitry •Cross-talk between sensor elements •Drift of sensor output •Charge amplifier required •Not suitable for static applications •Not stretchable •Signal alteration and attenuation due to bending or misalignment •No cross-talk between wiring •POFs: flexible and durable •LEDs: high spatial resolution and low-cost •LEDs can be used both as transmitter and detector •Insensitive to electromagnetic radiation response of 2%/N to applied normal forces in the range of 0–0.6 N. The authors also present smaller arrays (4 × 4) with larger sensor size (2 mm × 2 mm) and lower spatial resolution in [44]. However, here the sensors can measure both normal and shear forces with a sensitivity of 2.1%/N and 0.5%/N, respectively, in a range of 0–2 N. In [45], Choi presents NiCr strain gauges that are also sensitive to normal and shear forces with a relatively high force sensitivity of 207 mV/N and 70 mV/N, respectively. In [46], high force sensitivity is traded for strength and durability by removing the thin diaphragm. Copper–nickel (CuNi) strain gauges are deposited on thin polyimide films which are attached to a polydimethylsiloxane (PDMS) membrane for flexibility. 8 × 8 arrays of 3 strain gauges each are presented with a spatial resolution of 4 mm and sensitivity in the order 10 mV/N and 0.5 mV/N for normal and shear forces, respectively. Embedding or covering micromachined sensor elements with an elastic material combines the advantages of MEMS with the mechanical flexibility of elastomers. In this way, MEMS strain gauges which are inherently brittle can be stretched and applied over curved surfaces and moveable joints. Moreover, the elastomer layer increases grasp quality and the robustness of the system. However, embedding leads to lower sensitivities, as well as the 178 H. Yousef et al. / Sensors and Actuators A 167 (2011) 171–187 Fig. 3. Common design of diaphragm based strain gauge sensors. A force applied on the surface of the foil or bump, the diaphragm deforms, in turn deforming the integrated strain gauges. Depending on the placement of the strain gauges (as well as the measurement method) it is possible to measure both normal and shear forces. transverse inverse problem, i.e., a sensory pattern registered by sensors inside or under an elastic material is not necessarily unique [5]. In [47,48], strain gauges are placed on orthogonally placed silicon-based microcantilevers embedded in a layer of PDMS or polyurethane. The strain gauge detects the deformation of both the cantilevers and elastomer due to applied stress. It is possible to discriminate between normal and shear stresses as the output of the individual cantilevers relative to each other will differ depending on the direction of the stress (see Fig. 4). Sensors covered with PDMS show a linear response to applied stress with a sensitivity of about 0.02%/N normal stress. The measurement range of the sensor is significantly lower for shear stresses (1 N versus 8 N). Sensors covered with polyurethane have a sensitivity around 30 times higher than in PDMS, but do not show a linear response to applied stress. In [49], the sensor is further developed to detect stress distribution and object shape and a 3 × 3 sensor array in an area of 3 mm × 3 mm is presented with a sensitivity of 2.2 mV/N and 0.14 mV/N in the normal and shear directions, respectively. The force sensitivity of embedded strain gauge sensors can be increased by introducing ridges on the surface of the soft material covering the sensors (resembling human epidermal ridges) [50,51]. The ridges enhance mechanical deformations due to applied forces. In addition, friction is increased leading to higher grasp stability. A sensor sensitivity increase of a factor 2 is demonstrated. Fig. 4. Structure and operation of embedded tilted cantilevers. Reproduced from [46] Copyright © 2007, IEEE. Fig. 5. (a) 3D force sensor array based on independent measurement of two standing cantilevers, one for each tangential direction, and a beam for normal forces. The sensor chips are embedded are embedded in PDMS. (b) An individual sensor chip. (c) Perpendicularly standing cantilever embedded in PDMS. (d) Beam for measurement of normal forces. Reproduced from [55] Copyright © 2009, IEEE. 3.1.2. Micromachined piezoresistors In piezoresistors, mechanical stress is detected by a change in resistance of the piezoresistive material itself. Piezoresistors in general have smaller lateral dimensions and can achieve a high output per area than strain gauges. Silicon and other semiconductor materials have high piezoresistive responses, but are however brittle and fragile. As with strain gauges, embedding them in an elastomer allows for mechanical flexibility, but decreases sensitivity and can introduce ambiguity (see discussion above). In [52] a silicon-based piezoresistive sensor is embedded directly into a soft fingertip. The sensor chip has 4 cross-beams with 18 piezoresistors on its surface for detecting longitudinal and shear stresses, with a sensitivity of 0.085 V/N in the normal direction and 0.039 V/N in the lateral directions. The total sensor package has lateral dimensions of 1 mm × 5 mm, and 3.3 mm in thickness. After packaging, the sensor chip is moulded into a polyurethane hemisphere representing a fingertip. The authors demonstrate high accuracy measurements for both pushing (vertical) and sliding (lateral). The direction and magnitude of shear forces are detected in [53] by independently measuring the change in resistance of two perpendicularly placed standing silicon-based cantilevers embedded in PDMS. The sensor sensitivity in the direction parallel to the applied force is a factor 20 higher than in the direction perpendicular to the force. In this way it is possible to distinguish the axial components of the applied force. Although the cantilever dimensions are in micrometer range, the sensor package has a total size of 20 mm × 20 mm range. To increase the shear force detection range, the cantilevers are embedded in a liquid-filled chamber that is added to the structure increasing the shear force in [54]. In this way, shear forces of up to 3 N can be applied to the sensor surface without damaging the cantilevers. In [55], a beam is added to the sensor configuration for detecting normal forces (see Fig. 5), but with a sensitivity that is a factor 10 lower than the sensitivity for H. Yousef et al. / Sensors and Actuators A 167 (2011) 171–187 shear forces. Here, the spatial resolution of the sensor package is also in the centimetre range. In [56], an embedded tri-axial silicon based piezoresistor sensor is bonded to a flex PCB, and the resulting package is encapsulated in polyurethane. The complete sensor package has diameter of 3 mm and a sensitivity of 0.1 V/N and 0.4 V/N for normal and shear forces, respectively. In [57], the same sensor is integrated with a flex PCB and an optical signal converter forming a flexible optoelectronic system that can be wrapped around a finger. In this way, the amount of wiring and cross-talk is envisioned to be reduced. The sensor system showed a pressure sensitivity of 1.7 kPa. In [58], the sensing range and sensitivity of an embedded piezoresistor sensor can be tuned. The sensor consists of four cantilevers with silicon piezoresistors which are embedded in a PDMS–cobalt composite material. By increasing the concentration of cobalt particles relative to PDMS, the stiffness of the polymer layer increases, which in turn increases the maximum load and sensing range of the sensor. This increase is however accompanied with a decrease in sensor sensitivity. Sensitivities ranging from 0.52% to 3.4% are presented for applied normal forces, and from 1.0% to 2.2% for applied shear forces. 3.1.3. Conductive polymers and fabrics Polymer films are mechanically flexible, robust, and can be chemically resistant. Furthermore, polymer-based sensors can be fabricated using large-area low-cost fabrication techniques such as roll-to-roll fabrication and screen printing [59]. A fully poly- 179 Fig. 6. (a) Schematic of cross-section of an IPMC material in undeformed state, i.e., uniform distribution of cations throughout the material. (b) The IPMC, (c) crosssectional view of Flemion-based sensor presented in [60]. A deformation of the Flemion layer results in a non-uniform distribution of cations, and consequently surface charge accumulation. The Flemion is deposited on a patterned electrodes on a PDMS bump, allowing for measurement of both shear and normal forces. Reprinted with permission from [60]. Copyright 2009, American Institute of Physics [63]. meric and mechanically flexible piezoresistive sensor is presented in [60]. Here, the sensing material consists of a porous nylon matrix which is filled with electrodeposited polypyrrole. The conductivity of the composite material increases with applied compressive load, and a flexible tactile sensor is presented with a stable sensitivity of 0.023%/kPa in an applied pressure range of 20–600 kPa. A 32 × 32 sensor array using this material is demonstrated in [61]. In [62], porous polyurethane is rendered conductive and pressure sensitive by polymerisation of pyrrole into the porous matrix. The material shows a linear response of around 0.016%/N to an applied compression force range of 0–35 N. In [63], Flemion, an ion-polymer metal composite (IPMC), is used as the sensing layer in a 3D tactile sensor (see Fig. 6). The mem- Fig. 7. (a) A rectangular cut of the conductive knit fabric fitted and stretched around a dummy human face. (b) Points showing where force is applied on the face. (c) The resulting EIT image of applied forces. Reproduced from [66] Copyright © 2007, IEEE. 180 H. Yousef et al. / Sensors and Actuators A 167 (2011) 171–187 Fig. 8. Pressure distribution measurements of solid stamps that are applied with a normal force onto the sensor arrays in [66]. The solid stamps are shown in (a–d), and their corresponding tactile images are shown in (e–h). Reproduced from [68], Copyright 2008, with permission from Elsevier. brane is deposited on a patterned electrode on a PDMS tactile bump. When an external force is applied to the bump, the Flemion layer is deformed causing an internal charge redistribution and hence an output potential. The sensor sensitivity in the normal direction is a factor 2 higher than in the lateral direction. The sensitivity of Flemion to an applied load was found to be an order of magnitude higher than Nafion, another commercial available IPMC. In another category of polymer-based resistive sensors, a sheet of a conductive polymer is sandwiched between two electrodes. The resistance of the interface between the conductive material and the electrodes (contact resistance) changes with applied load, and hence such setups can be used as tactile sensors [64]. A few conductive polymers and their use in tactile sensing are reviewed in [64]. The authors also present a sensor using a layer of Ethyl Vinyl Acetate (EVA). In [62] a flexible sensor is presented using a layer of commercially available Velostat (3MTM ) sandwiched between two polyimide foils with the electrode patterns [65]. In [66,67] Electrical Impedance Tomography (EIT) is used to image the resistance distribution of a layer of conductive fabric due to applied pressure. An array of electrodes is connected around a conductive fabric that is stretched over the face and body of a humanoid robot. By applying a current between the electrodes, the current flows over the whole conductive fabric resulting in an electrical potential distribution based on the resistance distribution of the material which in turn is a result of applied pressure (see Fig. 7). The system has a point-to-point spatial resolution of 9 mm and is used to detect forces up to 20 N. The authors also show that the developed conductive fabric can be stretched to higher degrees and show less hysteresis than conductive elastomers. 3.1.4. Conductive elastomer composites A common choice of pressure sensitive material is elastomers that are enriched with conductive filler particles. When an external force is applied to the sensor deforming the elastomer composite layer, its resistivity changes depending on the type of conductive particles, their volume percentage in the elastomer and the resulting material stiffness. As elastomers are highly stretchable they make excellent candidates for application on curved surfaces and moving parts. Moreover, the use of a soft material mimics human skin and increases grasp quality. However, applications are mainly restricted to pressure sensing as the materials conduct isotropically. Further disadvantages are that the sensors suffer from hysteresis and low dynamic ranges. In [68], drops of a conductive elastomer are dispensed directly onto electrodes on the surface of a flex PCB forming an 8 × 8 sensor array. By applying separated drops of polymer instead of a full layer, cross-talk between the sensors is decreased. Furthermore, the elec- trode pattern includes structures that act as temperature sensing pads. Solid stamps are pressed onto the array with normal forces of 5–10 N, and the resulting pressure distributions are presented as tactile images (see Fig. 8) with a spatial resolution of 5 mm. In [69], larger arrays (32 × 32) with comparable spatial and pressure resolution are presented with the use of pads of electrically conductive transfer tape. In [70], drops of conductive polymer are directly dispensed in the intersection points of a mesh of spiral copper electrodes. The spiral electrodes consist of copper wires that are wound around nylon lines (see Fig. 9) allowing for higher degrees of stretching and bending. An 8 × 8 sensor array is presented in an area of 20 mm × 20 mm. Tactile images of solid stamps applied with a pressure of 450 kPa and spatial resolution of 3 mm are shown. In [71], the arrays are expanded to a 16 × 16 array covering an area of 160 mm × 160 mm on the arm of a mannequin. The tactile sensing elements show the same sensitivity as in [70] but with considerably lower spatial resolution. In [72], a network of horizontal and vertical wires is stitched into a layer of conductive rubber. An array of sensing elements is formed at the intersections of the rows and columns. In this way, the mechanical flexibility of the rubber is utilised while at the same time avoiding deformation and delamination of the elastomer layer due to excessive tangential forces. The sensor elements show a repeatable but non-linear and hysteric response to applied pressure in the range of 0–200 kPa. The authors present an array of 16 × 3 sensors in an area of 44 mm × 12 mm. Someya et al. argue that as the number and complexity of pressure sensor arrays increases, the switching matrix in the acquisition electronics cannot be realized with conventional silicon-based transistors without loosing mechanical flexibility [73]. This is Fig. 9. (a) The fabricated extendable spiral electrode. (b) The sensor array stretched over a ping pong ball. Reproduced from [70] Copyright © 2009, IEEE. H. Yousef et al. / Sensors and Actuators A 167 (2011) 171–187 Fig. 10. (a) Flexible PDMS-based pressure sensor network with an integrated OFET matrix used for switching and as read-out electronics. Reproduced from [73], Copyright 2004, National Academy of Sciences, U.S.A. (b) Mechanically processed net material containing pressure and temperature sensor network with integrated OFETs. The net material is stretched over an egg. Reproduced from [75], Copyright 2005, National Academy of Sciences, U.S.A. solved by integrating arrays of flexible organic field-effect transistors (OFETs) into a PDMS-based pressure sensing layer. OFETs are deposited onto flexible plastic foils using large-area, low cost techniques. An array of 32 × 32 sensors with integrated OFETs is successfully demonstrated with a spatial resolution of 1 mm (see Fig. 10a). The sensor output is in the ␮A range to applied pressures of 0–30 kPa. A concept for scaling up the sensor arrays for larger area coverage is presented [74]. In [75], the plastic film containing the OFET structures is mechanically processed to form a highly stretchable net material (see Fig. 10b) allowing a stretch of 25%. In addition, temperature sensors are also included in the array. The presented net material is mechanically weak but is strong enough for applications requiring a single stretch, i.e., without recurring flexing. The sensor output and sensing range are comparable to [73]. 3.1.5. Conductive fluids In [76], a finger structure mimicking a human finger is presented consisting of a rigid core with a layer of sensing electrodes on its surface. A weakly conductive fluid is sandwiched between the core and the outer elastomeric skin layer. When the outer layer is pressed, the fluid path around the electrodes is deformed, resulting in a change in impedance. The resulting impedance pattern gives an indication about the direction and magnitude of the force, the point of contact and object shape. As the elastomer layer is part of the sensing structure, the authors argue it is not an impediment to the sensing quality as compared to, e.g. embedded sensors. The sensor system has a large force sensitivity range of 0.01–40 N with impedances ranging from 5 k to 1000 k. The sensor output is dependent on the shape and contact area of the probe applying the force. To avoid ambiguity the shape of the contacting object must be known prior to contact. Alternatively, the shape of the object can be 181 Fig. 11. The triangle module. (a) Each sensor implements 12 taxels and hosts the capacitive transduction electronics. (b) The thick layer of silicone rubber foam covering the sensors and the conductive layer used as ground plane sprayed on top. Reproduced from [79] Copyright © 2008, IEEE. deduced by active exploration, i.e., exploration over time and comparing results to expected values based on previous experience, hence, similar to how the human haptic system works. The spatial resolution is expected to be in the mm range. In [77], a thermistor is added to the sensor system for measurement of temperature and heat fluxes related to the material properties of the objects in contact. 3.2. Capacitive sensors Capacitive tactile sensing is one of the most sensitive techniques for detecting small deflections of structures without direct temperature dependence [6]. In [78], Pritchard et al. demonstrate arrays of capacitive sensors that are fabricated directly on flexible thin films of polyimide with thicknesses down to 25 ␮m. Here, each capacitive sensor consists of two circular evaporated gold plates with an intermediate parylene dielectric layer. The sensors show a linear response to applied pressure, and arrays of 5 sensors with 500 ␮m diameter and 1 mm pitch give an output between 0.02 and 0.04 pF for an applied pressure of 700 kPa. The nominal values of the sensors increase with repetitive loading. The authors propose to solve this with data processing. In [79], mechanically flexible modules containing a complete sensor and communication system are presented. By combining several modules, large areas, such as the entire body of a robot, can be covered. In the prototypes, capacitance to digital converter integrated circuits (CDCs) are integrated on one side of a flex PCB, and circular copper capacitor plates serving as sensing units (taxels) on the other (see Fig. 11). A layer of silicone rubber is applied onto the side containing the taxels. This layer is covered with a layer of spray-on conductive silicon rubber that acts as the ground plane. When pressure is applied onto the ground plane, the deformation of the silicon rubber changes the capacitance of the circuit which is measured by the CDCs. Measurements on 2 such taxels are presented in a range of −0.4 N to 0.3 N. In [80], this sensing principle 182 H. Yousef et al. / Sensors and Actuators A 167 (2011) 171–187 Fig. 12. Principle of operation to measure normal and shear stress. (a) Cross-sectional view of a tactile cell without applied forces. (b) Response to normal force. (c) Response to shear force. Reproduced from [81] Copyright © 2008, IEEE. is used on a prototype finger. Here, the sensors are not fabricated in modules, but as a cover for fingertips. The sensor electronics are integrated in a rigid PCB which is incorporated in the bottom side of the fingertip. By substituting the rigid substrate with a flexible one, the sensor system can possibly be used to cover the entire circumference of a finger. Measurements show a nonlinear response to applied pressure, with higher sensitivity for lower pressures. In the aforementioned sensors, each of the capacitors consists of two parallel plates, and only forces that are normal to the surface can be measured. By modifying the design of the sensor plates, and/or the measurement electronics, the same sensing principles can be used to measure shear forces as well. In [81], Lee et al. present a configuration of parallel plate capacitors that enables sensing in both tangential and normal directions. The system is wholly embedded in PDMS. A PDMS spacer layer with air gaps is found between the capacitor plates. When an external force is applied, the air gaps are deformed leading to a change in capacitance. Each sensor consists of four pairs of plates, and the pattern of changes in capacitance of the plates gives a measure of the magnitude and direction of the applied force (see Fig. 12). The total cell has a width of 2 mm. An array of 8 × 8 cells is demonstrated in a measurement range of 0–10 mN with sensitivities between 2.5%/mN and 3.0%/mN in both normal and tangential directions (with capacitances in the fF/mN range). The authors show that the sensing range can be increased by increasing the height of the air gap between the capacitor plates. This however results in a decrease in the output capacitance. Normal and shear force maps are presented for an array of 4 × 4 sensors. The issues of shielding and cross-talk between the different cells in each sensor, or between the different sensors in the array, are not addressed. By reconfiguring the acquisition electronics in [82], the authors show that the capacitive sensors can also be used for proximity sensing. Arrays of 16 × 16 sensors are presented with dual-mode tactile and proximity sensing, In this case, the tactile sensing is only for normal forces. da Rocha et al. present another configuration of plates for measuring both vertical and horizontal contact forces [83]. Each sensor consists of four variable capacitors that share the same top electrode. As the applied force deforms the dielectric material, the area of each of the bottom electrodes that is covered by the common top electrode varies, and hence so does the capacitance of each. The read-out capacitances of the system of capacitors determine the magnitude and direction of the applied forces. A proof of principle of the system is presented, however, the applied forces and sensitivity are not characterised. The dimensions of the capacitors are in the centimetre range. Cross-talk and shielding issues are not treated. To reduce wiring in tactile skins, Hoshi and Shinoda propose what they have named a cell-bridge system [84]. Here, each cell is a capacitive sensor consisting of two capacitors that are formed by alternating layers of conductive fabric and dielectric material. A network of signal transmission devices (bridges) is embedded in the material. The ‘bridges’ communicate with each other through the conductive layers in the ‘cell’ material, reducing wiring. A completely wireless capacitive based pressure sensor is presented by Shinoda and Oasa in [85]. Here, passive resonators are embedded in a layer of silicone rubber. Each resonator consists of a capacitor and a coil that is inductively coupled to a ground coil that is located on the outside layer of the sensor. An applied stress causes a change of capacitance of the embedded capacitor which in turn causes a shift in the resonance frequency of the LC resonator. This shift is read out by the ground coil. In [86], alternating layers of metal and dielectric material are deposited on an elastic hollow tube to form a stretchable fabric-like capacitive sensor. The hollow fibres are woven into a mechanically deformable 2D mesh, where each intersection point between two fibres forms a sensor. By interweaving the hollow fibres with cotton threads, the authors present an array of 4 × 4 sensors with a spatial resolution down to 2 mm. The authors argue that the spatial resolution can be reduced by using commercial weaving machines. 3.3. Piezoelectric sensors Piezoelectric sensors convert an applied stress or force into an electric voltage [87]. Piezoelectric sensors are highly sensitive with high voltage outputs even to small deformation. The sensing elements do not require a supply of electrical power, and hence the sensors are considered to be highly reliable and can be applied to a wide range of applications. The voltage output however decreases over time and piezoelectric sensors are therefore only suitable for detecting dynamic forces [8]. Polyvinylidene fluoride (PVDF) films are common piezoelectric materials in tactile sensing applications due to their mechanical flexibility, high piezoelectric coefficients, dimensional stability, low weight and chemical inertness [88,89]. H. Yousef et al. / Sensors and Actuators A 167 (2011) 171–187 183 Fig. 13. Cross sectional sketch of embedded strain gauge and PVDF film receptors: the fingertip consists of a metal bar, a body, and a skin layer inspired by the structure of the human finger. The body and the skin layer are made of different types of silicon rubber. Strain gauges and PVDF films are randomly embedded in the fingertip as receptors. Reprinted from [90], Copyright 2006, with permission from Elsevier. In [90], PVDF film sensors are fabricated separately and embedded into silicone layer which is moulded onto a robotic fingertip (see Fig. 13). Strain gauges are also embedded, and the two different sensors’ function and distribution the human finger’s tactile receptors. The PVDF sensors gives an output of around 1 V during rubbing and pushing of different textures, while the strain gauges have an output between 0.5 and 1 V. In [91], the sensor system is further developed to a prototype for tactile skin for flat areas such as the palm of the hand. The presented sensor principle shows potential for biomimetic artificial skin, with the ability to sense texture, and possibly with further development, forces. Limitations may lay in fabrication constraints when moving to larger areas such a full anthropomorphic hand. Furthermore, as each of the sensors in both fingertip and palm skin is connected by a wire, the number of and complexity of the wiring will possibly become too large for a full hand unless resolution is sacrificed. In [92,93] arrays of piezoelectric sensors sense and partially process at the same site, mimicking the mechanoreceptors in human skin. Two different sensor arrays are presented. In the first, an array of 32 microelectrodes is adhered to a PVDF composite film. The microelectrodes have a radius of 500 ␮m and a pitch of 1 mm. The sensor array shows a linear response to applied forces in the range of 0.02–4 N. The output depends on the thickness of the PVDF film that is used, with a sensitivity of 0.2 V/N and 0.4 V/N for 25 ␮m and 50 ␮m thick films, respectively. The authors show that there is significant cross-talk between the sensors (20%). In the second type [93], arrays of field emitting transistors (FETs) are coated with a piezoelectric polymer, forming piezoelectric oxide semiconductor field effect transistors (POSFET). In this way, the transducer and processing circuitry are included in the same entity, decreasing processing time and reducing cross-talk. The taxels are 1 mm × 1 mm large and have a linear response in the range of 0.2–5 N with a sensitivity of 0.5 V/N. 3.4. Optical sensors As the number of sensors increases in the tactile skin, wiring complexity and cross-talk become an issue when electrical signals are used. A solution is to use fibre optic cables to carry signals. With the introduction of plastic optical fibres (POFs), previous limitations of rigidity and fragility are overcome [57]. A POF-based microbend optical fibre sensor is presented in [94]. A 2-D mesh of fibres is embedded in a silicone elastomer. The optical measuring system consists of an LED light source and a CCD detector (see Fig. 14). When a contact force is applied to the mesh, the POFs bend, modifying the light intensity. The sensor shows a linear response to applied forces of up to 15 N with a resolution of 0.05 N. However, the sensor suffers from hysteresis errors due to the material properties of the silicone rubber. Fig. 14. Fabricated prototype of optical fibre tactile sensors; its flexibility is demonstrated. Reproduced from [94] Copyright © 2008, IEEE. In [95], a complete optical sensing system with integrated POFs is presented (see Fig. 15). A rigid transparent finger base is covered with a silicone gel that serves as the skin. Steel reflector chips are integrated onto the top layer of the silicone skin, and bundles of POFs are embedded perpendicularly through the finger frame, one under each reflector chip. When an object makes contact with the surface of the skin, the reflector chips on the skin surface change their position modifying the light that is collected by the POFs. The location and shape of objects in contact can be calculated with sub-millimetre resolution. The magnitude and direction of applied forces can be derived from measurements if the material dimensions and properties of the skin layer are known. Rossiter and Mukai argue that by using light emitting diodes (LEDs) as both light transmitters and detectors, the bulk and complexity of fibre optic cables can be avoided [96,97]. In this way the system is simplified as only one type of active device is needed. Furthermore, LEDs are smaller, cheaper and can be mounted with high physical resolution. The authors present a sensor in which two LEDs are embedded in a deformable, semi-opaque medium that acts as the skin layer. One LED emits light to the upper surface of the skin, while the other detects the light that is reflected back. When a contact force is applied, the skin deforms and the amount of light that reaches the detector LED decreases. The authors present a 4 × 4 matrix with a sensitivity in the range of mV/N and range up to 6 N. Ohmura et al. present an 8 × 4 array LED based sensors mounted on flexible foils with a sensitivity in the range of mV/N [98]. The sensors are covered by polyurethane foam that deforms under applied forces. The flexible foil is cut into branch-like shapes Fig. 15. Schematic of the location-sensing skin. Reproduced from [95] Copyright © 2008, IEEE. 184 H. Yousef et al. / Sensors and Actuators A 167 (2011) 171–187 of the sensing pixel is completely removed. However, a camera is required for each of the digits. 3.5. Organic field-effect transistors (OFETs) as sensors Fig. 16. LED-based sensors mounted on a flexible foil that is cut into finger-like shapes to facilitate bending around complex curved surfaces. Reproduced from [97] Copyright © 2006, IEEE. that can be bent so that the relative distance between the sensors can be adapted (see Fig. 16). In this way, the flexible substrate can conform to more complex curved surfaces than are possible with an unstructured flexible foil. Moreover, the authors show how large areas can be covered by cut-and-paste of individual modules. In [99], two LEDs of different wavelengths (blue and red) are mounted, one on top of the other, under an elastic urethane membrane which represents the skin. When a contact force is applied to the membrane, the pattern of the light under the membrane is altered and the position, angle of incidence, and magnitude of force can be calculated (see Fig. 17). In [100], silicone rubber markers, red and blue, are embedded in a silicon rubber fingertip in two layers, one for each colour. A camera mounted inside the robot finger structure captures an image of the displacement of the markers which enables the calculation of the applied force vector field. Forces of 0.2–2 N were measured in all three directions with a force resolution of 0.3 N and a spatial resolution of 5 mm. By using direct imaging instead of markers and inverse calculations, resolution can be improved. In [101], directly imaged force vector field images are presented with a force resolution of up to 0.05 N and a spatial resolution of 5 mm. In both systems [100,101], electrical wiring to each Fig. 17. Relationship between way of irradiation of light and circular area formed by the light. Reproduced from [99] Copyright © 2008, IEEE. Darlinski et al. show that pentacene OFETs are directly affected by applied mechanical pressure and can be directly used as pressure sensing elements [102]. The authors argue that this simplifies the sensor fabrication technique as there is no need for further process steps. In the demonstrated prototype, the OFETs are deposited on a rigid substrate. Manunza et al. present pentacene OFETs where the transistors perform both pressure sensing and switching [103,104]. The prototypes are completely mechanically flexible as they are fabricated on a 1.6 ␮m thick Mylar foil. Here, the Mylar acts both as the gate dielectric and the carrier substrate for mechanical support. The sensors show a linear current response (I/I) of 0.07/kPa applied pressure. In [105], researchers from the same group present 3 × 3 arrays of the pentacene OFETs sensors, with a spatial resolution of 7 mm. The sensors in the array can be switched on independently. In [106], Mannsfeld et al. present arrays of OFETs as capacitive pressure sensors. The output current of the OFET is directly dependant to their capacitance. Hence, by using an elastomer that mechanically deforms under pressure as the dielectric layer, the OFET can be used as a pressure sensor. Here, the dielectric layer consists of a thin film of PDMS. The authors show that by microstructuring the PDMS layer into micrometer-sized pyramids, the sensitivity is increased by a factor 30 and the relaxation time of the sensor is significantly reduced. The presented sensors are highly sensitive to pressures under 2 kPa with a sensitivity of 1 ␮A/kPa. For higher pressures (2–18 kPa), the sensitivity is around 0.3 ␮A/kPa. 4. A comparison of sensor solutions and sensing techniques Tactile sensor solutions reviewed above that fulfil, or in our opinion can be developed to fulfil, the functional and technical specifications for in-hand manipulation defined in Section 2.3 are presented in Table 2. The different sensing solutions are compared with respect to sensitivity, range, spatial resolution, size, and mechanical flexibility, and the solution in each sensor category that is found to have the highest performance for a specific parameter is highlighted in bold. Here, fully stretchable, solutions are found to be the most advantageous. As can be seen in Table 2, an analytical comparison of the sensitivity and range is in essence not possible given the variation in how the sensors are characterised and which parameters are presented in the literature. However, for 3D force sensors, a ratio comparing the ratio of sensitivity to normal forces to sensitivity to shear forces is introduced as this can be seen as a measure of the “usability” of the sensor as a 3D force sensing solution. Taking the sensors’ size/spatial resolution into account, suggestions for the suitable area of application on a robotic hand are presented for the different sensor solutions. Finally, to emphasize the importance of distributed tactile sensing, the sensors are presented in categories of arrayed and non-arrayed solutions. A comparison of the performance of sensors that make use of the same sensing principle does not reveal strong common tendencies. This can be attributed to the fact that in general each of the presented solutions is developed for solving specific problems for different applications, and hence has its own set of advantages and limitations. Nevertheless, observations on the general advantages and disadvantages of the different sensing techniques can be made and are presented in Table 3. This table, in combination with Table 2, can be used as a tool for choosing an appropriate sensing technique for a particular application. H. Yousef et al. / Sensors and Actuators A 167 (2011) 171–187 5. Summary and conclusions A review of the state-of-the-art tactile sensing solutions that are suitable for dextrous in-hand manipulation shows that resistive sensing techniques are still the predominant choice for tactile sensing and a multitude of different resistive techniques exist. In addition, several publications can also be found on piezoelectric and capacitive sensing techniques, as well as on different optical systems. New techniques such as Electrical Impedance Tomography (EIT) and the use of embedded passive coils have been introduced. Moreover, a number of new materials such as ion-polymer metal composites (IPMCs), organic field emitting transistors (OFETs), and novel conductive materials have been introduced for increased sensitivity, functionality and performance. Continuous developments in the fields of materials engineering, nanotechnology and fabrication technologies can lead to advances in sensor performance, as well as reliability and mechanical properties. Several potential improvements in sensor performance are discussed in [6]. For example, the introduction of nanofeatures such as carbon nanotubes (CNTs), nano-coils and nanowires into, e.g. resistive sensors based on conductive elastomer composites, replacing conventional conductive particles, can greatly improve sensitivity and force range. In addition, due to the axial sensitivity of, e.g. CNTs to rotation, such conductive elastomer composites can also possibly be used in measurements of shear stress during manipulation. Recent advances in stretchable organic and printed electronics show that fully stretchable distributed tactile sensing can be achieved, allowing improved coverage of arbitrary surfaces, using low cost fabrication techniques [107,108]. In addition to the development of sensing techniques, packaging and integration with the rest of the robotic system have received a considerable amount of attention. Here, an important goal is to reduce the amount and complexity of wiring to increase robustness and reduce cross-talk. The main approaches have been found to be: encapsulation of sensor elements directly onto flex PCBs, including flexible/stretchable wiring in the sensor structure, and direct integration of processing and communication transistors into the sensor array itself. A growing interest can also be found for integrating tactile sensing with other modalities, in particular temperature, to increase the functionality of the dexterous robotic manipulator. Here, it also found that increased functionality without increasing wiring complexity is an important driving force. A tendency within the robotics scientific community is to emulate the human sense of touch with respect to the structure, physiological properties and functionality of human skin, particularly in the fingertips. Looking at the high spatial resolution, multimodality, and varied functionality of human skin, this may seem to be unfeasible. However, artificial skin is developed for specific applications such as in-hand manipulation. Taking this into account, specific subgroups of the functionalities of the human sense of touch can be mimicked to design successful tactile sensing systems. Furthermore, as artificial tactile sensing is not limited by some of the factors that limit the human sense, other sensitivities, dynamic ranges, functionalities, temporal resolutions can be introduced. In addition, the success of a tactile sensing system is also related to the post-processing of the collected data. Even though the tactile sensing hardware today may be limited with regard to dimensions, distribution and functionality, the robotic manipulation system as a whole can be improved, possibly achieving human-like manipulation, by further development of tactile information processing software. An example is presented [109] where cellular neural network computing combines sensory information from the tactile sensor array in the artificial skin presented in [57] with proprioceptive sensory data from the robot hand structure to detect events during manipulation, and to provide real-time stable 185 grasping capabilities for objects with unknown shape and surface properties. In this way, limitations of the tactile hardware can be overcome by software. There is a large understanding of the physiology of the human skin as well as, more recently, the neurophysiology of the sense of touch and grasping, and a large amount of work has been presented on the replication of grasping for robotic manipulation. More complicated dexterous manipulation tasks, such as in-hand manipulation, have however not been studied in the same extent be it within neuroscience, cognitive physiology or robotics. This can be attributed to the immense complexity associated with even the simplest in-hand manipulation, and that the execution of such movements in humans are also dependant on processes other than sensory feedback such as learning, planning and the perception of object affordances, e.g. [110–112]. Hence, we see that a true understanding of human dexterous manipulation is still lacking, and consequently, we find that a full understanding of the optimal design of tactile skins for intelligent robotic manipulation has also still not been achieved. The advent of reliable, distributed tactile sensing solutions will revolutionise intelligent manipulation for robotic hands including artificial anthropomorphic hands. Although as shown above, tactile sensor technology has reached quite a level of maturity, currently available sensors cannot handle the tactile sensing requirements of those modern robot hands that are intended for advanced and, possibly, human-like object handling tasks, such as in-hand manipulation. Since existing sensor technologies can provide only a small subset of the needed tactile sensory information (e.g. limited force range, insufficient spatial and temporal resolution, limited sensing area and limited capability of sensing shear forces), today’s control approaches for robotic hands are based on traditional control methods, involving kinematic robot models and complicated offline planning strategies and often requiring the use of additional, external sensors such as vision. Integrating highly distributed tactile sensing capabilities with articulated robot hands will allow the creation of truly reactive manipulation devices that can handle objects with ease in the presence of uncertainty, modelling inaccuracies, non-linear interaction dynamics and unpredictable mechanical object properties. Acknowledgments This work has received financial support from the HANDLE project which is funded by European Community’s Seventh Framework Programme (FP7/2007–2013) under grant agreement ICT 231640. Dr. Moustapha Hafez, Dr. Margarita Anastassovsa, Dr. Josè Lozada and Professor Lakmal Seneviratne are acknowledged for valuable input on the manuscript. References [1] A. Vernick, Turning Science Fiction into Scientific Reality. Cynthia Breazeal Talks About Women and Technology, 2006, Available at: http://www2.scholastic.com/browse/article.jsp?id=7739 (accessed 18.09.09). [2] K.E. Pennywitt, Robotic tactile sensing, Byte 11 (1) (1986) 177–200. [3] J. Tegin, J. Wikander, Tactile sensing in intelligent robotic manipulation – a review, Ind. Robot 32 (2005) 64–70. [4] HANDLE Project Website, 2009, Available at: http://www.handle-project.eu/ (accessed 18.09.09). [5] M.H. Lee, H.R. Nicholls, Review article tactile sensing for mechatronics—a state of the art survey, Mechatronics 9 (February) (1999) 1–31. [6] V. Maheshwari, R. Saraf, Tactile devices to sense touch on a par with a human finger, Angew. Chem. Int. Ed. 47 (2008) 7808–7826. [7] D. De Rossi, F. Carpi, E.P. Scilingo, Polymer based interfaces as bioinspired ‘smart skins’, Adv. Colloid Interface Sci. 116 (November) (2005) 165–178. [8] P. Puangmali, K. Althoefer, L. Seneviratne, D. Murphy, P. Dasgupta, State-ofthe-art in force and tactile sensing for minimally invasive surgery, IEEE Sens. J. 8 (2008) 371–381. 186 H. Yousef et al. / Sensors and Actuators A 167 (2011) 171–187 [9] J. Dargahi, S. Najarian, Advances in tactile sensors design/manufacturing and its impact on robotics applications—a review, Ind. Robot 32 (2005) 268–281. [10] R.S. Dahiya, G. Metta, M. Valle, G. Sandini, Tactile sensing—from humans to humanoids, IEEE Trans. Robot. 26 (1) (2010) 1–20. [11] M. Kimmerle, L. Mainwaring, M. Borenstein, The functional repertoire of the hand and its application to assessment, Am. J. Occup. Ther. 57 (2003) 489–498. [12] M. Cutkosky, On grasp choice, grasp models and the design of hands for manufacturing tasks, IEEE Trans. Robot. Autom. 5 (1989) 269–279. [13] A. Bicchi, V. Kumar, Robotic grasping and contact: a review, in: Proceedings of IEEE International Conference on Robotics and Automation (ICRA), Vol. 1, 2000, pp. 348–353. [14] C. Choi, M. Shin, S. Kwon, W. Park, J. Kim, Understanding of hands and task characteristics for development of biomimetic robot hands, in: Proceedings of the 8th IEEE-RAS International Conference on Humanoid Robot (Humanoids), 2008, pp. 413–417. [15] C. Edussooriya, H. Hapuachchi, D. Rajiv, R. Ranasinghe, S. Munasinghe, Analysis of grasping and slip detection of the human hand, in: Proceedings of the 4th International Conference on Information and Automation for Sustainability (ICIAFS), 2008, pp. 261–266. [16] M. Ciocarlie, P. Allen, Hand posture subspaces for dexterous robotic grasping, Int. J. Robot. Res. 28 (July) (2009) 851–867. [17] S. Cobos, M. Ferre, M. Ángel Sánchez-Urán, J. Ortego, R. Aracil, Human hand descriptions and gesture recognition for object manipulation, Comput. Methods Biomech. Biomed. Eng. 13 (2010) 305. [18] K. Pont, M. Wallen, A. Bundy, J. Case-Smith, Reliability and validity of the test of in-hand manipulation in children ages 5 to 6 years, Am. J. Occup. Ther. 62 (4) (2008) 384–392. [19] J.P. Piek, G.B. Baynam, N.C. Barret, The relationship between fine and gross motor ability, self-perceptions and self-worth in children and adolescents, Hum. Movement Sci. 25 (1) (2006) 65–75. [20] K.M. Deutsch, K.M. Newell, Noise, variability and the development of children’s perceptual-motor skills, Dev. Rev. 25 (2) (2005) 155–180. [21] J. Summers, D. Larkin, D. Dewey, Activities of daily living in children with developmental coordination disorder: dressing, personal hygiene and eating skills, Hum. Movement Sci. 27 (2) (2008) 215–229. [22] J.P. Piek, L. Dawson, L.M. Smith, N. Gasson, The role of early fine and gross motor development on later motor and cognitive ability, Hum. Movement Sci. 27 (5) (2008) 668–681. [23] J.M. Elliott, K. Connolly, A classification of manipulative hand movements, Dev. Med. Child Neurol. 26 (3) (1984) 283–296. [24] C.E. Exner, In-hand manipulation skills, in: J. Case-Smith, C. Pehoski (Eds.), Development of hand skills in the child, American Occupational Therapy Association, 1992, pp. 35–45 (Rockville, MD, USA). [25] K. Pont, M. Wallen, A. Bundy, Conceptualising a modified system for classification of in-hand manipulation, Aust. Occup. Ther. J. 56 (2009) 2–15. [26] R.S. Johansson, J.R. Flanagan, Coding and use of tactile signals from the fingertips in object manipulation tasks, Nat. Rev. Neurosci. 10 (May) (2009) 345–359. [27] J.R. Flanagan, A.M. Wing, The role of internal models in motion planning and control: evidence from grip force adjustments during movements of handheld loads, J. Neurosci. 17 (1997) 1519–1528. [28] V.M. Zatsiorsky, M.L. Latash, Multi-finger prehensionan overview, J. Motor Behav. 40 (2008) 446–476. [29] J.R. Martin, M.L. Latash, V.M. Zatsiorsky, Interaction of finger enslaving and error compensation in multiple finger force production, Exp. Brain Res. 192 (2009) 293–298. [30] M. Santello, M. Flanders, J.F. Soechting, Postural hand synergies for tool use, J. Neurosci. 18 (1998) 10105–10115. [31] R.S. Johansson, G. Westling, Roles of glabrous skin receptors and sensorimotor memory in automatic control of precision grip when lifting rougher or more slippery objects, Exp. Brain Res. 56 (October) (1984) 550–564. [32] R. Johansson, J. Flanagan, Tactile sensory control of object manipulation in humans, in: E. Gardner, J.H. Kaas (Eds.), The Senses: A Comprehensive Reference. Volume 6: Somatosensation, Academic Press, San Diego, CA, USA, 2008, pp. 67–86. [33] M. Knibestöl, Å.B. Vallbo, Single unit analysis of mechanoreceptor activity from the human glabrous skin, Acta Physiol. Scand. 80 (1970) 178–195. [34] J. Dargahi, S. Najarian, Human tactile perception as a standard for artificial tactile sensing—a review, Int. J. Med. Robot. 1 (June) (2004) 23–35. [35] B. Edin, L. Ascari, L. Beccai, S. Roccella, J. Cabibihan, M. Carrozza, Bio-inspired sensorization of a biomechatronic robot hand for the grasp-and-lift task, Brain Res. Bull. 75 (April) (2008) 785–795. [36] J.A. Paradiso, J. Lifton, M. Broxton, Sensate media; multimodal electronic skins as dense sensor networks, BT Technol. J. 22 (2004) 32–44. [37] B. Gery, S. Gottschlich, A tactile sensing system for dexterous manipulation, in: Proceedings of the IEEE/RSJ/GI International Conference on Intelligent Robots and Systems (IROS), Vol. 2, 1994, pp. 1384–1390. [38] Y. Xu, F. Jiang, S. Newbern, A. Huang, C. Ho, Y. Tai, Flexible shear-stress sensor skin and its application to unmanned aerial vehicles, Sens. Actuators A: Phys. 105 (August) (2003) 321–329. [39] Y. Xu, Y.-C. Tai, A. Huang, C.-M. Ho, IC-integrated flexible shear-stress sensor skin, J. Microelectromech. Syst. 12 (2003) 740–747. [40] R.B. Katragadda, Y. Xu, A novel intelligent textile technology based on silicon flexible skins, Sens. Actuators A: Phys. 143 (May) (2008) 169–174. [41] R.B. Katragadda, Y. Xu, A novel intelligent textile technology based on silicon flexible skins, in: Proceedings. 9th IEEE International Symposium on Wearable Computer, 2005, pp. 78–81. [42] K. Kim, K.R. Lee, W.H. Kim, K. Park, T. Kim, J. Kim, J.J. Pak, Polymer-based flexible tactile sensor up to 32 × 32 arrays integrated with interconnection terminals, Sens. Actuators A: Phys. 156 (December) (2009) 284–291. [43] J. Engel, J. Chen, C. Liu, Development of polyimide flexible tactile sensor skin, J. Micromech. Microeng. 13 (May) (2003) 359–366. [44] K. Kim, K.R. Lee, D.S. Lee, N.-K. Cho, W.H. Kim, K.-B. Park, H.-D. Park, Y.K. Kim, Y.-K. Park, J.-H. Kim, A silicon-based flexible tactile sensor for ubiquitous robot companion applications, J. Phys. Conf. Ser. 34 (2006) 399–403. [45] W.-C. Choi, Polymer micromachined flexible tactile sensor for three-axial loads detection, Trans. Electr. Electron. Mater. 11 (3) (2010) 130–133. [46] E.-S. Hwang, J.-H. Seo, Y.-J. Kim, A polymer-based flexible tactile sensor for both normal and shear load detections and its application for robotics, J. Microelectromech. Syst. 16 (3) (2007) 556–563. [47] Y. Huang, M. Sohgawa, K. Yamashita, T. Kanashima, M. Okuyama, M. Noda, H. Noma, Fabrication and normal/shear stress responses of tactile sensors of polymer/Si cantilevers embedded in PDMS and urethane gel elastomers, IEEJ Trans. Sens. Micromachines 128 (2008) 193–197. [48] M. Sohgawa, Y. Huang, K. Yamashita, T. Kanashima, M. Noda, M. Okuyama, H. Noma, Fabrication and characterization of silicon-polymer beam structures for cantilever-type tactile sensors, in: Proceedings of the International Conference on Solid-State Sensors, Actuators and Microsystems (TRANSDUCERS), 2007, pp. 1461–1464. [49] M. Sohgawa, T. Mima, H. Onishi, T. Kanashima, M. Okuyama, K. Yamashita, M. Noda, M. Higuchi, H. Noma, Tactile array sensor with inclined chromium/silicon piezoresistive cantilevers embedded in elastomer, in: Proceedings of the International Conference on Solid-State Sensors, Actuators and Microsystems Conference (TRANSDUCERS), 2009, pp. 284–287. [50] Y. Zhang, Sensitivity enhancement of a micro-scale biomimetic tactile sensor with epidermal ridges, J. Micromech. Microeng. 20 (2010) 085012. [51] Y. Zhang, Y. Mukaibo, T. Maeno, A multi-purpose tactile sensor inspired by human finger for texture and tissue stiffness detection, in: Proceedings of the IEEE International Conference on Robotics and Biomimetics (ROBIO), 2006, pp. 159–164. [52] V.A. Ho, D.V. Dao, S. Sugiyama, S. Hirai, Analysis of sliding of a soft fingertip embedded with a novel micro force/moment sensor: simulation, experiment, and application, in: Proceedings of the IEEE International Conference on Robotics and Automation (ICRA), 2009, pp. 889–894. [53] K. Noda, K. Hoshino, K. Matsumoto, I. Shimoyama, A shear stress sensor for tactile sensing with the piezoresistive cantilever standing in elastic material, Sens. Actuators A: Phys. 127 (March) (2006) 295–301. [54] K. Noda, K. Matsumoto, I. Shimoyama, Flexible tactile sensor sheet with liquid filter for shear force detection, in: Proceedings of the IEEE 22nd International Conference on Micro Electro Mechanical Systems (MEMS), Sorrento, Italy, 2009, pp. 785–788. [55] K. Noda, Y. Hashimoto, Y. Tanaka, I. Shimoyama, MEMS on robot applications, in: Proceedings of the Solid-State Sensors, Actuators and Microsystems Conference (TRANSDUCERS). International, 2009, pp. 2176–2181. [56] L. Beccai, S. Roccella, L. Ascari, P. Valdastri, A. Sieber, M. Carrozza, P. Dario, Development and experimental analysis of a soft compliant tactile microsensor for anthropomorphic artificial hand, IEEE ASME Trans. Mechatron. 13 (2008) 158–168. [57] L. Ascari, P. Corradi, L. Beccai, C. Laschi, A miniaturized and flexible optoelectronic sensing system for tactile skin, J. Micromech. Microeng. 17 (November) (2007) 2288–2298. [58] C. Wen, W. Fang, Tuning the sensing range and sensitivity of three axes tactile sensors using the polymer composite membrane, Sens. Actuators A: Phys. 145–146 (2008) 14–22. [59] C. Liu, Recent developments in polymer MEMS, Adv. Mater. 19 (2007) 3783–3790. [60] L.C. Tsao, D.R. Chang, W.P. Shih, K.C. Fan, Fabrication and characterization of electro-active polymer for flexible tactile sensing array, Key Eng. Mater. 381–382 (2008) 391–394. [61] S. Yu, D. Chang, L. Tsao, W. Shih, P. Chang, Porous nylon with electro-active dopants as flexible sensors and actuators, in: Proceedings of the IEEE 21st International Conference on Micro Electro Mechanical Systems (MEMS), Tucson, AZ, USA, 2008, pp. 908–911. [62] S. Brady, K.T. Lau, W. Megill, G.G. Wallace, D. Diamond, The development and characterisation of conducting polymeric-based sensing device, Synth. Met. 154 (1–3) (2005) 25–28. [63] J. Wang, H. Sato, C. Xu, M. Taya, Bioinspired design of tactile sensors based on Flemion, J. Appl. Phys. 105 (April) (2009) 083515–83517. [64] K. Weiss, H. Worn, The working principle of resistive tactile sensor cells, in: Proceedings of the IEEE International Conference on Mechatronics and Automation, Vol. 1, 2005, pp. 471–476. [65] Z. Del Prete, L. Monteleone, R. Steindler, A novel pressure array sensor based on contact resistance variation: metrological properties, Rev. Sci. Instrum. 72 (February) (2001) 1548–1553. [66] H. Alirezaei, A. Nagakubo, Y. Kuniyoshi, A highly stretchable tactile distribution sensor for smooth surfaced humanoids, in: Proceedings of the 7th IEEE-RAS International Conference on Humanoid Robots (Humanoids), 2007, pp. 167–173. H. Yousef et al. / Sensors and Actuators A 167 (2011) 171–187 [67] H. Alirezaei, A. Nagakubo, Y. Kuniyoshi, A tactile distribution sensor which enables stable measurement under high and dynamic stretch, in: Proceedings of the IEEE Symposium on 3D User Interfaces (3DUI), 2009, pp. 87–93. [68] Y.-J. Yang, M.-Y. Cheng, W.-Y. Chang, L.-C. Tsao, S.-A. Yang, W.-P. Shih, F.-Y. Chang, S.-H. Chang, K.-C. Fan, An integrated flexible temperature and tactile sensing array using PI-copper films, Sens. Actuators A: Phys. 143 (May) (2008) 143–153. [69] Y.-J. Yang, M.-Y. Cheng, S.-C. Shih, X.-H. Huang, C.-M. Tsao, F.-Y. Chang, K.-C. Fan, A 32 × 32 temperature and tactile sensing array using PI-copper films, Int. J. Adv. Manuf. Technol. 46 (2010) 945–956. [70] M.-Y. Cheng, C.-M. Tsao, Y. Lai, Y.-J. Yang, A novel highly-twistable tactile sensing array using extendable spiral electrodes, in: Proceedings of the IEEE 22nd International Conference on Micro Electro Mechanical Systems (MEMS), 2009, pp. 92–95. [71] M.-Y. Cheng, C.-M. Tsao, Y.-J. Yang, An anthropomorphic robotic skin using highly twistable tactile sensing array, in: Proceedings of the IEEE Conference on Industrial Electronics and Applications (ICIEA), 2010, pp. 650–655. [72] M. Shimojo, A. Namiki, M. Ishikawa, R. Makino, K. Mabuchi, A tactile sensor sheet using pressure conductive rubber with electrical-wires stitched method, IEEE Sens. J. 4 (2004) 589–596. [73] T. Someya, T. Sekitani, S. Iba, Y. Kato, H. Kawaguchi, T. Sakurai, A large-area, flexible pressure sensor matrix with organic field-effect transistors for artificial skin applications, Proc. Natl. Acad. Sci. U.S.A. 101 (July) (2004) 9966–9970. [74] H. Kawaguchi, T. Someya, T. Sekitani, T. Sakurai, Cut-and-paste customization of organic FET integrated circuit and its application to electronic artificial skin, IEEE J. Solid-State Circ. 40 (1) (2005) 177–185. [75] T. Someya, Y. Kato, T. Sekitani, S. Iba, Y. Noguchi, Y. Murase, H. Kawaguchi, T. Sakurai, Conformable, flexible, large-area networks of pressure and thermal sensors with organic transistor active matrixes, Proc. Natl. Acad. Sci. U.S.A. 102 (August) (2005) 12321–12325. [76] N. Wettels, V. Santos, R. Johansson, G. Loeb, Biomimetic tactile sensor array, Adv. Robot. 22 (2008) 829–849. [77] C.H. Lin, T.W. Erickson, J.A. Fischel, N. Wettels, G.E. Loeb, Signal processing and fabrication of a biomimetic tactile sensor array with thermal, force and microvibration modalities, in: Proceedings of the IEEE International Conference on Robotics and Biomimetics (ROBIO), 2009, pp. 129–134. [78] E. Pritchard, M. Mahfouz, B. Evans, S. Eliza, M. Haider, Flexible capacitive sensors for high resolution pressure measurement, in: Proceedings of the IEEE 7th Conference on Sensors (IEEE-sensors), 2008, pp. 1484–1487. [79] G. Cannata, M. Maggiali, G. Metta, G. Sandini, An embedded artificial skin for humanoid robots, in: Proceedings of the IEEE International Conference on Multisensor Fusion and Integration for Intelligent Systems (MFI), 2008, pp. 434–438. [80] A. Schmitz, M. Maggiali, M. Randazzo, L. Natale, G. Metta, A prototype fingertip with high spatial resolution pressure sensing for the robot iCub, in: Proceedings of the 8th IEEE-RAS International Conference on Humanoid Robots (Humanoids), 2008, pp. 423–428. [81] H. Lee, J. Chung, S. Chang, E. Yoon, Normal and shear force measurement using a flexible polymer tactile sensor with embedded multiple capacitors, J. Microelectromech. Syst. 17 (2008) 934–942. [82] H. Lee, S. Chang, E. Yoon, Dual-mode capacitive proximity sensor for robot application: implementation of tactile and proximity sensing capability on a single polymer platform using shared electrodes, IEEE Sens. J. 9 (2009) 1748–1755. [83] J.G.V. da Rocha, S. Lanceros-Mendez, Capacitive sensor for three-axis force measurements and its readout electronics, IEEE Trans. Instrum. Meas. 58 (2009) 2830–2836. [84] T. Hoshi, H. Shinoda, A Large area robot skin based on cell-bridge system, in: Proceedings of the IEEE 5th Conference on Sensors, 2006, pp. 827–830. [85] H. Shinoda, H. Oasa, Wireless tactile sensing element using stress-sensitive resonator, IEEE ASME Trans. Mechatron. 5 (2000) 258–265. [86] Y. Hasegawa, M. Shikida, D. Ogura, Y. Suzuki, K. Sato, Fabrication of a wearable fabric tactile sensor produced by artificial hollow fiber, J. Micromech. Microeng. 18 (2008) 085014. [87] C. Steinem, A. Janshoff, Piezoelectric Sensors, Springer-Verlag, Berlin/Heidelberg, Germany, 2007. [88] P. Ueberschlag, PVDF piezoelectric polymer, Sens. Rev. 21 (2001) 118–125. [89] S. Lang, S. Muensit, Review of some lesser-known applications of piezoelectric and pyroelectric polymers, Appl. Phys. A: Mater. Sci. Process. 85 (November) (2006) 125–134. [90] K. Hosoda, Y. Tada, M. Asada, Anthropomorphic robotic soft fingertip with randomly distributed receptors, Robot. Auton. Syst. 54 (February) (2006) 104–109. [91] S. Takamuku, G. Gomez, K. Hosoda, R. Pfeifer, Haptic discrimination of material properties by a robotic hand, in: Proceedings of the IEEE 6th International Conference on, Development and Learning (ICDL 2007), 2007, pp. 1–6. [92] R.S. Dahiya, M. Valle, G. Metta, L. Lorenzelli, C. Collini, Tactile sensing arrays for humanoid robots, in: Proceedings of the Ph IEEE-PRIME’07, The 3rd Conference on PhD research on Microelectronics and Electronics, 2007, pp. 201–204. [93] R.S. Dahiya, M. Valle, G. Metta, L. Lorenzelli, Bio-inspired tactile sensing arrays, in: Proceedings of Bioengineered and Bioinspired Systems IV, Dresden, Germany, SPIE, 2009, 73650D-9. [94] J.-S. Heo, J.-Y. Kim, J.-J. Lee, Tactile sensors using the distributed optical fiber sensors, in: Proceedings of the 3rd International Conference on Sensing Technology (ICST), 2008, pp. 486–490. 187 [95] Y. Yamada, M. Morizono, U. Umetani, T. Takahashi, Highly soft viscoelastic robot skin with a contact object-location-sensing capability, IEEE Trans. Ind. Electron. 52 (2005) 960–968. [96] J. Rossiter, T. Mukai, A novel tactile sensor using a matrix of LEDs operating in both photoemitter and photodetector modes, in: Proceedings of the IEEE 4th International Conference on Sensors (IEEE-Sensors), vol. 52, October 31–November 1, 2005, pp. 960–968. [97] J. Rossiter, T. Mukai, An LED-based tactile sensor for multi-sensing over large areas, in: Proceedings of the IEEE 5th International Conference on Sensors (IEEE-Sensors), 2006, pp. 835–838. [98] Y. Ohmura, Y. Kuniyoshi, A. Nagakubo, Conformable and scalable tactile sensor skin for curved surfaces, in: Proceedings of the IEEE International Conference on Robotics and Automation (ICRA), Vol. 1, 2006, pp. 1348–1353. [99] K. Hoshino, D. Mori, Three-dimensional tactile sensor with thin and soft elastic body, in: Proceedings of the IEEE Workshop on Advanced robotics and Its Social Impacts, ARSO 2008, 2008, pp. 1–6. [100] K. Sato, K. Kamiyama, H. Nii, N. Kawakami, S. Tachi, Measurement of force vector field of robotic finger using vision-based haptic sensor, in: Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2008), Nice, France, 2008, pp. 488–493. [101] C. Chorley, C. Melhuish, T. Pipe, J. Rossiter, Development of a tactile sensor based on biologically inspired edge encoding, in: Proceedings of the International Conference on Adv Robot (ICAR), Munich, Germany, 2009, pp. 1–6. [102] G. Darlinski, U. Bottger, R. Waser, H. Klauk, M. Halik, U. Zschieschang, G. Schmid, C. Dehm, Mechanical force sensors using organic thin-film transistors, J. Appl. Phys. 97 (May) (2005) 093708–93714. [103] I. Manunza, A. Sulis, A. Bonfiglio, Pressure sensing by flexible, organic, field effect transistors, Appl. Phys. Lett. 89 (October) (2006) 143502–143503. [104] I. Manunza, A. Bonfiglio, Pressure sensing using a completely flexible organic transistor, Biosens. Bioelectron. 22 (June) (2007) 2775–2779. [105] P. Cosseddu, A. Bonfiglio, R. Neelgund, H.W. Tyrer, Arrays of pressure sensors based on organic field effect: a new perspective for non invasive monitoring, in: Proceedings of the 31st Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), 2009, pp. 6151–6154. [106] S.C.B. Mannsfeld, B.C. Tee, R.M. Stoltenberg, C.V.H. Chen, S. Barman, B.V.O. Muir, A.N. Sokolov, C. Reese, Z. Bao, Highly sensitive flexible pressure sensors with microstructured rubber dielectric layers, Nat. Mater. (2010), Advance online publication (accessed 21.09.10). [107] T. Sekitani, T. Someya, Stretchable, large-area organic electronics, Adv. Mater. 22 (2010) 2228–2246. [108] T. Sekitani, H. Nakajima, H. Maeda, T. Fukushima, T. Aida, K. Hata, T. Someya, Stretchable active-matrix organic light-emitting diode display using printable elastic conductors, Nat. Mater. 8 (2009) 494–499. [109] L. Ascari, U. Bertocchi, P. Corradi, C. Laschi, P. Dario, Bio-inspired grasp control in a robotic hand with massive sensorial input, Biol. Cybern. 100 (2008) 109–128. [110] K.S. Lashley, The problem of serial order in behavior, in: L.A. Jeffress (Ed.), Cerebral Mechanisms in Behavior, John Wiley & Sons, New York, NY, USA, 1951, pp. 112–146. [111] J. Gibson, The Ecological Approach to Visual Perception, Lawrence Erlbaum Associates Publishers, Mahwah, NJ, USA, 1979. [112] E. Oztop, M. Arbib, Schema design and implementation of grasp-related mirror neuron system, Biol. Cybern. 87 (2002) 116–140. Biographies Hanna Yousef received her M.Sc. degree in applied physics from the Norwegian University of Technology and Science (NTNU), Norway, in 2001 and the Ph.D. degree in Microsystems Technology from Uppsala University, Sweden, in 2008. In 2008 she was R&D project manager at Rolling Optics AB, Sweden, developing novel holographic materials. She is currently at the Sensory and Ambient Interfaces Laboratory at CEA LIST as a researcher and project manager. Her fields of interest include microsystems design and fabrication process technologies for flexible materials allowing for new applications in sensorics, robotics, and optics. Mehdi Boukallel received his M.S. degree in automation and micromechanics from Université de Franche-Comté, France in 2000 and the Ph.D. degree in robotics and automation from the Université de Franche-Comté, France in 2003. From 2003 to 2004 he was as a researcher staff member with the Microrobotics Group at Besançon, France. He worked from 2005 to 2007 as Postdoctoral researcher at the Université Pierre et Marie Curie, Paris, France in the micro and nanomanipulation group. Since 2007 he works as a research engineer, at the French Atomic Energy Commission CEALIST. His research interests include microrobotics design, biomanipulation, sensors design and control of smart actuators. Kaspar Althoefer received his first degree in electronic engineering from the University of Aachen, Germany, and the Ph.D. degree in electronic engineering from King’s College London, U.K. He is currently a Reader at King’s College London. He has extensive expertise in the areas of robot-based applications, sensing, and embedded intelligence. He has published over 160 refereed research papers related to mechatronics and robotics.

RELATED PAPERS

RELATED TOPICS

Log In

Tactile sensing for dexterous in-hand manipulation in robotics—A review

Tactile sensing for dexterous in-hand manipulation in robotics—A review

Related Papers

RELATED PAPERS

RELATED TOPICS