[go: up one dir, main page]
Next Article in Journal
Stabilize and Flatten Multi-Wavelength Erbium-Doped Fiber Laser through Accurate Hybrid Dual-Ring-Configuration Control
Previous Article in Journal
A Three-Axis Magnetic Field Microsensor Fabricated Utilizing a CMOS Process
 
 
Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
Journals
Applied Sciences
Volume 7
Issue 12
10.3390/app7121291
applsci-logo
Submit to this Journal Review for this Journal Propose a Special Issue
► Article Menu

Article Menu

share Share announcement Help format_quote Cite question_answer Discuss in SciProfiles
first_page
settings
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Vibrotactile Display of Flight Attitude with Combination of Multiple Coding Parameters

by
Qiangqiang Ouyang
†,
Juan Wu
*,† and
Miao Wu
†
Robotic Sensor and Control Laboratory, School of Instrument Science and Engineering, Southeast University, Sipailou No. 2, Nanjing 210096, China
*
Author to whom correspondence should be addressed.
†
These authors contributed equally to this work.
Appl. Sci. 2017, 7(12), 1291; https://doi.org/10.3390/app7121291
Submission received: 7 November 2017 / Revised: 1 December 2017 / Accepted: 4 December 2017 / Published: 12 December 2017
Browse Figures
Figure 1
<p>Vibrotactile actuator.</p> "> Figure 2
<p>The arrangement of distribution of tactors around the torso and prototype of hlthe designed vibrotactile vest. In frontal of body, five tactors are located on the lower abdomen, upper abdomen, slightly below the chest, middle of chest and below of the clavicle respectively. In back of body, five tactors are located on the hip above, central spine, right shoulder, left shoulder, and cervical spine respectively. In the left or right side of body, five tactors, from down to up, are located on hipbone above, rib, oxter, chest and shoulder respectively.</p> "> Figure 3
<p>Overview of vibrotactile system for cueing flight attitude. (<b>a</b>) System block diagram of vibrotactile TSAS and Tactor arrangement around the torso of pilot. The controller receive flight attitude information from the geomagnetic sensor and activate vibrotactile vest to vibrate in the pattern corresponding to the attitude; (<b>b</b>) Schematic diagram of encoding flight states for different tactor columns; (<b>c</b>) Definition of Flight attitude information of aircraft.</p> "> Figure 4
<p>Schematic diagram of tactile apparent motion.</p> "> Figure 5
<p>A subject during the experiment.</p> "> Figure 6
<p>Schematic diagram of coding methods with combination of two parameters: (<b>a</b>) location and intensity (LI); (<b>b</b>) location and rhythm(LR); (<b>c</b>) location andmode(LM). For the convenience of labelling angel intervals, five degrees in vertical coordinate indicates interval [5, 10) in Table 3, 10 indicates [10, 15), and so forth. The cueing time is maximum time to convey a vibrotactile pattern. Different colour blocks represent tactors locating at different sites in a column.</p> "> Figure 7
<p>Recognition accuracy and reaction time (s) for each coding method in Experiment 1. Error bars indicate standard error.</p> "> Figure 8
<p>Schematic diagram of coding methods with combination of three parameters: (<b>a</b>) location, rhythm of times and intensity (LRI1); (<b>b</b>) location, rhythm of duration and intensity (LRI2); (<b>c</b>) location, rhythm of times and intensity (LRI3); (<b>d</b>) location, rhythm and mode (LRM). For the convenience of labelling angel intervals, five degrees in vertical coordinate indicates the interval [5, 10) in <a href="#applsci-07-01291-t003" class="html-table">Table 3</a>, 10 indicates [10, 15) and so forth.</p> "> Figure 9
<p>Recognition accuracy and reaction time (s) for each coding method in experiment 2. Error bars indicate standard error.</p> "> Figure 10
<p>Information transfer rate for different coding methods; the error bars indicate standard error.</p> "> Figure 11
<p>The curve of recognition accuracy change with set angle in preferred coding method. Error bars indicate standard error.</p> ">
Versions Notes

Abstract

:
Vibrotactile (vibratory tactile) displays have been reported as effective in enhancing awareness of flight attitude for pilots and releasing other heavily loaded sensory channels. Although some work has been done on vibrotactile coding of flight attitude, there is lack of a systematic investigation into coding methods with combination of multiple coding parameters. In this paper, seven coding methods with seven combinations of multiple coding parameters (location, rhythm, intensity, and mode) were systematically studied to cue flight attitude for pilots with vibrotactile vest. We conducted two psychophysical experiments in a static environment in which the attitude information in the form of vibrotactile feedback are presented randomly, and quantitatively evaluated the effectiveness of the vest according to the users’ recognition accuracy, reaction time and information transfer rate. The results show that vibrotactile vest is effective to cue attitude information. The preferred coding method with combinations of location, rhythm and mode allowed users to perform with lowest reaction time and highest recognition accuracy and yield about 255 bits/min of information transfer rate. Overall, the presented work provides valuable insights and guidance for the design of vibrotactile aids for the pilots.

1. Introduction

Flying an aircraft, especially the fighter plane, is a challenging activity and exposes the pilot to many potential hazards. One of the most significant of these is spatial disorientation (SD), which is a term used to describe a variety of incidents occurring in flight where the pilot fails to sense attitude, or motion of one’s own aircraft relative to the earth or other significant objects [1]. In aviation accidents and incidents, SD accounts for some six to 32 percent of major accidents, and some 15 to 69 percent of fatal accidents [1].
There are several factors causing SD problem such as visual and vestibular illusions, fatigue driving, instrument fault, and environmental factors (flight in rain, above sea, through cloud) [1,2,3]. The decline in situation awareness of pilots due to these factors is the direct reason to cause SD problem [1]. Traditional aircraft instruments, such as gyro-stabilized altitude indicator, HUD, SVS (synthetic visual system), and EVS (enhanced visual system), provide pilots the situation of aircraft via visual or audio channel [4]. However, the traditional instruments occupy additional audio-visual resources [5]. Pilots need a high concentration of the auditory and visual channel in accessing information from the vicinity for safe ambulation (e.g., approaching objects and acoustic and visual signals from instruments and alerts), and the two channels will not be available for other cues. Additionally, the pilot cannot keep the situation of aircraft to a suitable standard via the two traditional senses when flying aircraft above the sea [2], through rain and cloud [3], or in a condition of altered gravitoinertial acceleration [6,7,8].
According to previous psychological studies, the issues above may be solved by using a free sensory channel liking tactile sense, for example, the multiple resource model of human information processing predicts no performance degradation when independent resources or information channels are used to present information (e.g., refer to [9,10]). The advantage of touch is it neither blocks the visual nor the auditory sense [11]. Employing a tactile channel may release other heavily loaded sensory channels, therefore potentially providing a major safety enhancement [12]. Due to the convenience and real-timeliness for changing the pattern of tactile stimulation, vibratory stimuli have been widely adopted in haptic displays.
Recently, favourable effects of vibrotactile displays on navigation performance, situational awareness, and workload reduction have been shown in many application, such as vehicle driving, guiding blind people, and particularly in the high workload group [13,14,15,16,17]. To date, very few studies have examined the implementation of vibrotactile displays for conveying flight attitude information of aircraft. The flight attitude is an important parameter for pilots to maintain their awareness of situation of aircraft [3,18]. During the flight, the pilots should ascertain within about five degrees of pitch and roll information of aircraft. It is required that the vibrotactile display provide multiple tactile patterns to the pilots [3]. The TSAS (tactile situation awareness system) consisted of 8 × 5 matrices of pneumatic tactors, developed by Rupert and colleagues is perhaps the most fully implemented and tested system. The resolution of encoding flight attitude for this TSAS is high, at about five degrees in fine flight [3]. In their coding methods, vibrotactile patterns with different vibration intensity were employed to cue precise angle information. However, coding methods with intensity did not produce intuitive vibrotactile patterns mapping corresponding angles, and need long periods of training to master the vibrotactile commands conveyed by the TSAS in practice.
The coding strategies for these vibrotactile displays have so far been rather basic [19]. An intuitive and well-perceivable coding method can reduce training time and increase user acceptance of a tactile device. According to previous work in enhancing pilots’ spatial awareness with vibrotactile devices, the parameters that can be used to encode flight information include three basic dimensions: spatial location, temporal rhythm and intensity of vibration [20]. However, systematic investigations into coding strategies for conveying flight attitude information with a combination of multiple coding parameters are exceptional rare in reported work. A combination of multiple coding parameters are essential to improve the effectiveness of vibrotactile display, since we can determine the vibrotactile coding capacity of each vibration parameters and differences between the perceptions of vibrations with different parameters. The test results of the vibrotactile display designed by [21,22] illustrated that the best way to encode information with tactile channel is using the coding parameter with as many dimensions as possible. For instance, spatiotemporal patterns will yield superior identification over spatial patterns and patterns encoded by a single motor’s intensity for an area of skin [23].
The goal of this work is to investigate preferred coding methods with a combination of multiple coding parameters for cueing precise directional information through vibrotactile feedback, using well perceivable and easily comprehensible vibrotactile patterns. The torso provides an extensive haptic space for presenting tactile information, with approximately half the total surface area of the body. The skin covering the torso is capable of precisely encoding information since it contains hundreds of mechanoreceptors [24,25]. In addition, a belt-type device can be worn under a coat without attract public attention. Therefore, we focus on vest-type device to convey vibrotactile feedback for flight attitude of aircraft, and systematically investigated the coding methods with combination of parameters: location, rhythm, intensity, and modal.

2. Materials and Methods

2.1. Vibrotactile Actuators

There are three types of actuator available for haptic display: electromagnetic (DC coin motor or linear motor), pneumatic, or direct electric tactor in TSAS. The direct electric tactor can evoke a strong tactile sensation, however the range between absolute threshold and pain is very small; the amplitude of pneumatic tactor is fixed [20]. We selected the DC coin motor over the other types due to its lightweight, small size, low price and easiness of adjusting the amplitude. In order to increase the perceived intensity in flight condition, two motors were overlapped as a vibrotactile unit. We used 20 KOTL C1234B016F coin vibration motors operating at 3V of voltage and about 140 Hz of frequency (http://www.kotl.cn/cn/default.aspx). The intensity of vibration was controlled by PWM duty. For the convenience of narration, we used the phrase “tactor” to represent the vibrotactile unit used in this study. The vibrotactile actuator is shown in Figure 1.

2.2. Design of Wearing Vest

The vibrotactile display was designed by taking the inspiration from TASA proposed by Rupert [3], which consists of eight columns of a five tactor array. Since there is significant discrimination between the perceptions of vibrotactile stimuli from the cardinal sides of torso [17,26,27,28] , four columns of tactors were set on the four cardinal sites of the vest. Each column consists of five tactors. To make the vibrotactile vest suitable for everyone, we selected the high elastic material as the cloth of vest. In order to facilitate the user’s localizing the vibrations from different tactors, the tactors should be arranged properly to make the distance of each adjacent pair as far as possible. After wearing the vest, the tactor approximate distribution of tactors around the torso is depicted in Figure 2.

2.3. Tactile Coding Parameters

A fundamental factor that may affect effectiveness of vibrotactile displays is how this altitude information is encoded [3]. In the this section, several coding parameters that can be used to cue flight attitude are discussed.
Flight attitude of an aircraft can be divided into four basic states: pitch-down state, pitch-up state, roll-left state and roll-right state. The angle in each state ranges from 0 to 90 degree. The schematic diagram of vibrotactile system, the method of vibrotactile displaying of different flight states, and the graphical definition of flight attitude are illustrated in Figure 3.
As seen in Figure 3b, the angle of pitch ( θ ) and roll ( φ ) were be defined as an included angle between the earth-surface inertial reference frame (O, x g , y g , z g ) and aircraft-body coordinate frame (O, x b , y b , z b ). As illustrated in Figure 3b, the state information of aircraft were cued by the vibration on corresponding column or combination of two columns of tactors. In order to embody the S-R (stimulus-response) compatibility of space [29], the frontal, back, left and right column of tactors were employed to cue angle in pitch-up, pitch-down, roll-left and roll-right state, respectively. When both pitch and roll angles need to be cued, the corresponding columns of tactors will simultaneously be activated to vibrate. As illustrated in Table 1, the angle in each flight state was divided into four flight statuses: normal flight ranged in [5, 50) with five degrees apart, acrobatic flight ranged in [50, 90) with 10 degrees apart, fine flight ranged in [0, 5), and emergency ≥90 degree.
In order to distinguish the steady flight from the fault of vibrotactile system, attitude in steady flight was encoded by the tactor placed at the bottom of the corresponding column. Since the aircraft is in steady flight status in most of time, this tactor was controlled to vibrate with weak intensity and short duration considering energy efficiency. An angle of more than 90 degree was cued by simultaneous vibration of all tactors in corresponding column. Thus, the range of angle needed to encode in each flight state is [5, 90) with 14 angle intervals.
The parameters that can be used to encode angle and flight state include spatial location, intensity, rhythm and mode of vibration, which can be classified into two categories according to the number of vibrating tactors as illustrated in Table 2.
Spatial characteristics of a vibrotactile sensation is determined by the location of vibration stimulus on a vibrotactile display [30]. Of the four coding parameters shown in Table 2, the parameter of location is therefore mandatory to encode attitude information. However, it is not enough to encode 56 (14 × 4) angle intervals with 20 tactors only using parameter of location. Thus, there are other parameters in Table 2. They should be selected to combine with location for cueing all the angle intervals. There are about three identifiable levels for intensity of vibration on torso according to the results according to our previous psychophysical test for actuators [31]. Thus, the identifiable levels of intensity can be used to cue angle information more precisely. For instance, the weak, middle and strong vibration intensity of a tactor can be employed to indicate the three different angle intervals of a group in Table 1. However, the coding capacity of cuing information with intensity is limited, since there are only three tactor amplitude intensities that can be easily identified [32].
The rhythm of vibration is a time-domain parameter that can also be used to improve the precision of cueing angle. As is similar with the parameter of intensity, different times or duration lengths of vibration can be used to cue different angle intervals in a group.
Activating multiple tactors in vibration modes is another useful coding strategy. The three main parameters that control the feeling of different vibration modes are the duration of stimulus (DOS), stimulus onset asynchrony (SOA) and the inter-stimulus interval (ISI) [33]. By adjusting the parameter of DOS and ISI, we can make multiple tactors to be perceived as different vibrating modes as illustrated in Figure 4. The simultaneous vibration is also called tactile funnelling illusion: when two tactors vibrate simultaneously on skin, the perceived vibration of a virtual tactor located between the tactors will be felt [34]. The continuous vibration is also called “vibrotactile apparent movement”: When activating two or more tactors sequentially with a certain timing, the stimulation point is perceived as if it is moving continuously from one location to another, although the real stimulating sites are discrete [35]. Discrete motion will be perceived when the adjacent tactors are activated sequentially with an inter-stimulus interval (ISI).
From what is described above, the flight attitude can be displayed using coding methods with combination of two parameters or three parameters in Table 2. Therefore, two psychophysical experiments were designed to investigate the preferred combination of two and three parameters respectively.

3. Psychophysical Experiments

The tactile experiments are static stimulating tests, where the subjects wearing the vibrotactile vest perceived vibrations encoding attitude information in a stationary condition (see Figure 5). The objective of the experiments is to investigate the preferred coding method with combination of multiple vibration parameters.

3.1. Participants

The participants in this study were 20 postgraduate students of aerospace (18 males and two female), randomly recruited from the Nanjing University of Aeronautics and Astronautics. Their average age was 24 years (SD = 3 years). Their average flight experience in flight simulation was 860 h (SD = 125 h). All participants gave informed consent and received compensation for their time (RMB 100 per participant).

3.2. Experimental Design

Independent variables: we employed a within-subjects repeated measures design, with coding methods (CMs) as a between-subject factor and flight states as a within-subject factor ( roll-left, roll-right, pitch-up and pitch-down state).
Dependent variables: recognition accuracy and reaction time were implemented to evaluate the effectiveness of vibrotactile belt for cueing angle information.
Recognition accuracy is defined as the percentage of correct recognition of vibrotactile commands. Under the condition of correct judgment for flight states, we classified an identification with a difference between a reported and set angle no more than five degrees as a correct recognition, otherwise it was a wrong recognition. Recognition accuracy is proportion of number of correct recognitions to total number of commands.
Reaction time defined as the elapsed time between finishing the display of the vibrotactile command and the moment when the subject has reported a given angle. It should be noted that the time spent on clicking some buttons for reporting the corresponding angles was not included in the reaction time (the time taken for clicking buttons can be recorded in experimental software).
Information transfer rate (bits/min): to statistically analyze the preference of coding methods, we further calculate the information transfer rate (ITR) as shown in Equation (1). The most popular method for ITR calculation in brain-computer interface (BCI) research was defined by Wolpaw et al. in 1998, which is a simplified computational model based on Shannon channel theory under several assumptions [36,37].
B t = 60 T ( log 2 N + P · log 2 P + log 2 ( 1 − P N − 1 ) 1 − P
Generally, B t in bits/min is used to indicate the ITR, N is the number of possible choices and P is the probability that the desired choice will be selected, also called target identification accuracy or classifier accuracy. T (seconds/symbol) is the time needed to convey each symbol. In the current study, N can be seen as the total number of angles (60), T can be seen as the reaction time for each vibrotactile pattern, and P is obviously the recognition accuracy. The ITR is more comprehensive than recognition accuracy or reaction time for taking account of both the two evaluating criterion.

3.3. General Procedures

At the beginning of experiments, subjects were prompted to sit comfortably on a chair. They were asked to wear headphones playing white noise (see Figure 5). The headphone was important to block out sounds from the vibrotactile display as well as the office environment. Before the experiment started, the objective and the function of the vest was explained. A total of 52 vibrotactile patterns encoding attitude angles except for the angles ranging from 0 to 5 degrees the angle of more than 90 degrees, which were presented in sequential order to familiarize the subjects with the meanings of the patterns. For each coding method, the subjects should receive training for 10 min before starting formal experiments.
In the formal experiment, 52 × 3 trials were presented for each coding method via an experimental software. Thus for each subject, a total number of 1092 (52 × 3 × 7) trials were recorded throughout the whole experiment. At the end of each trial, subjects needed to report the flight attitude angles by clicking corresponding radio button in the experimental software. To avoid fatigue, participants could take a break between trials and sessions on request. In order to avoid biased responses, angles were presented in a pseudo-randomized order. The subjects needed to select the direction within the prescribed time (6 s). To reduce the practice and habituation error easily occurring in a psychological experiment [38], a random vibration pattern different from before was given to the subject when a judgement timed-out occurred. During the formal experiment, no feedback about correctness of the answers was given. The subject’s selected angle and reaction time were recorded in a database. After the completion of each coding method, subjects were asked to complete a questionnaire and provide a ranking of their preferred vibrotactile coding method.
To assess the impact of the coding method on the performance, we analyzed the data by employing a one-way within-subjects design ANOVA, using standard software (IBM SPSS statistics 20, IBM Corp., New York, NY, USA). The within-subjects factor is the coding methods. Simple main effects comparisons and post hoc analysis with Bonferroni correction were further performed to test the significant interaction effects. Wilcoxon signed rank tests and Friedman tests were conducted to compare the preferences among coding methods. We employed a one-way repeated measure multivariate analysis of variance (MANOVA) to test our objective measures.

3.4. Experiment 1

The Experiment 1 was conducted to find the preferred coding methods with a combination of two parameters.

3.4.1. Vibrotactile Coding Design

Since the parameter of vibrating location is mandatory in cueing information, there were three different combinations with two parameters as illustrated in Figure 6.
LI coding method. The angles within group shown in Table 1 was cued by vibration patterns with different intensity as illustrated in Figure 6a. The most obvious merit of this coding method is that the cueing time is short (300 ms).
LR coding method. The main idea of the second rendering method is to activate different times of corresponding tactor, depending on the angles within the group, as depicted in Figure 6b. The ISI and DOS was set to 60 ms and 150 ms, respectively, for each tactor at all angle levels.
LM coding method. In the third coding method, the angles within the group were encoded by activate adjacent pair of tactors in different vibrating mode (simultaneous, continuous and discrete) as illustrated Figure 6c. As is similar with the coding method LR, the same intensity was applied to all angle levels.
Overall, the intensity, rhythm and mode is combined with lactation in LI, LR, and LM coding method, respectively, to cue all the angles in each flight state.

3.4.2. Experimental Results

The results of Experiment 1 are depicted in Figure 7.
To quantitatively analyse the preference of combinations in term of recognition accuracy, post hoc tests with Bonferroni were performed.
As seen in Table 3, both LM and LR perform better than LI with significant difference (Z = −0.3580, p = 0.000), (Z = −0.2420, p = 0.005), respectively). LR perform better than LM but without a significant difference (Z = 0.1160, p = 0.122). As seen from the Figure 6, the cueing time of LR is shorter than that of LM. Overall, it can be obtained that the LR is the preferred combination of two vibration parameters.

3.5. Experiment 2

In the Experiment 1, we have determined that the LR (location and rhythm) is the preferred coding method of cueing attitude. In order to study whether there is improvement on performance when adding another vibration parameter at the basis of LR coding method, another experiment was carried to determine the preferred combinations of three vibration parameters.

3.5.1. Vibrotactile Coding Design

To enhance the discrimination between angle groups in Table 1, the vibration intensity or mode was employed as an additional coding parameter to facilitate the memorizing and mastering of the coding methods, and improve recognition accuracy.
LRI1 coding method combines with the parameters of location, rhythm and intensity. Compared to the LR coding method, the LRI1 coding method takes vibrating intensity as the additional coding parameter to enhance the discrimination between groups. As similar with LRI1, the LRI2 also uses vibrating intensity as the additional coding parameter, but implemented the rhythm with a different duration of stimulus as the main coding parameter. As similar with LRI1, the LRI3 also make use of vibration times to indicate different angles within the group, but the cueing time for all the angles are same. In order to facilitate the memorizing of vibrotactile patterns mapping angles, vibrating mode was employed as an additional coding parameter to distinguish between normal and acrobatic flight in the LRM method. Figure 8 shows the schematic diagram of coding methods with combination of three parameters.

3.5.2. Experimental Results

The average recognition accuracy and reaction time from all total trials was about 95 percent (SD:11 percent) and 1.02s (SD: 0.52 s) as illustrated in Figure 9. It can be seen that in all conditions, the coding method with combination of three parameters performed better than that of two parameters. To quantitatively analyze the preference of combinations in terms of recognition accuracy, post hoc tests with Bonferroni were performed.
In order to intuitively compare the performances of all the coding methods considering both recognition accuracy and reaction time, we further carry a ITR analysis as illustrated in Figure 10.
As seen in Figure 10, there is a significant effect of CM on ITR, and LRM has the highest ITR among the proposed coding methods, which is in accordance with results of ANOVA analysis (Table 4).
In order to further improve the recognition accuracy of the vibrotactile display. We carry a three-way repeated measure MANOVA to study whether there is significant difference of performance between levels of each coding parameter in the preferred coding method (LRM). There are four independent variables: flight states (four levels), locations (five levels), rhythms (two levels), and one dependent variable: recognition accuracy.
As seen in Table 4, there are no significant difference of performance between levels of each coding parameter, except for the parameter of location ( F ( 19 , 4 ) = 2.97, p = 0.032). In order to further improve the design of vibrotactile device and its coding strategy. We analysis the correct percentage changing with the set angle as shown in Figure 11.
As seen in Figure 11, the average recognition accuracy of all the angles exceeded 90%. The recognition accuracy at the angles ranging from 20 to 40 degrees and 60 degrees appear to be greater in terms of volatility than other angles. These angles were cued by tactor P2, P3, and P4, respectively (see Figure 2). The questionnaire also indicates that most of the subjects reported that they have difficulty in discriminating vibrations from these tactors, which is in accordance with the result of Table 5. It suggests that the setting sites for these tactors should be adjusted to further improve the perceptual performance of the vest.

4. Discussion

In this study, we have designed a vibrotactile vest to cue attitude information of aircraft for pilots and systematically investigated the effectiveness of vibrotactile coding methods with a combination of multiple vibration parameters.
The experimental data shows that, in general, the participants were positive about the use of vibrotactile vest to provide flight attitude. The experimental results also indicate that the rhythm performed better than coding parameters of intensity and mode (see Table 3). The average reaction time for the preferred coding method (LRM) is about 0.9 s, which meets the requirements of reducing pilots’ spatial disorientation with tactile command. In the current study, subjects received training for only 10 min, thus the presented vibrotactile vest can be further improved on the reaction time to which the pilots respond to the vibrotactile patterns.
It should be noted that there may be more discrimination between the different vibrotactile patterns if participants were driving a real aircraft instead of being seated, due to the continuous change of direction in practice. Besides the design of the current vibrotactile vest, it is convenient to wear for people of different waist sizes.

4.1. Comparisons with Previous Work

As a review of previous TSASs successfully implemented in aviation, there are several types of vibrotactile displays to cue situation information for pilots [3,18,39]. The comparison between their and our work on tactor cued is illustrated in Table 6.
The TSAS developed by the US Navy consists of 40 tactors, and was successfully implemented to cue flight attitude with a resolution of five degrees [3]. In this TSAS, different attitude angle ranges were cued by different tactors, and fine angle intervals in each range were indicated by corresponding intensity levels (i.e., LI coding method). The belt-type TSAS consisted of eight tactors, designed by Cardin et al. and can only be used to indicate the four basic flight states as shown in Figure 3b. In this TSAS, different flight states were indicated by different vibration locations [18]. As similar with the belt-type TSAS, the TSAS in literature [39] utilized the 60 torso tactors, with each ring mapped to at least 30 degree of the vertical dimension and each column mapped to 30 degree of the horizontal dimension. The TSAS in current work present attitude information of a similar resolution with the TSAS in literature [3] but using less tactors. Although some TSASs in reported work were also used to cue other situational information such as flight height and drifting direction, which we did not include in the current work, this information can be presented by improving the coding strategies in our current vibrotactile design. It will be discussed in following section.

4.2. Limitations of our Work

Although the TSAS in our current work is successfully implemented to cue attitude situation information and yield a resolution of five degrees, but not indicate other situational information such as flight height, drifting direction and velocity, which is also important to enhance the situation awareness and the reduce sensory workload during flight [20,40]. Fortunately, it is feasible to display integrated situation information by improving the coding strategies without changing the hardware design of the current vibrotactile system. For instance, the current vest consists of five rings of four-tactor arrays, the tactor rings from down to up can just be used to display five levels of flight height: low altitude flight, hollow flight, high-altitude flight, and ultrahigh altitude flight, respectively. Besides, the funnel tactile illusion can be implemented in each ring to cue a precise drifting direction and the horizontal velocity.

5. Conclusions

In the current study, we have systematically investigated the coding methods with combinations of multiple vibration parameters to convey attitude information. The results of the laboratory study show that our vibrotactile belt can achieve 91 percent of localization accuracy and enable pilots to receive vibrotactile attitude information with a resolution within five degrees. Further work is required to validate the effectiveness of the vibrotactile vest in practical application. A virtual environment equipped a virtual-reality helmet will be constructed to simulate flight. We will further test our tactile on a rotating room, where subjects can able to experience altered gravitoinertial acceleration (GIA). We will also improve coding strategies to cue integrated situation information for pilots with this vibrotactile vest. Overall, this work provides new evidence that the vest haptic displays have promise as an intuitive means of displaying the navigation signals of aircraft and may improve spatial awareness in low visibility environments.

Acknowledgments

The authors would like to express their gratitude to all the participants of this study, the authors would like to thank all the participants for their help in collecting data for this project. This research was supported in part by the Natural Science Foundation of China under grants 61473088.

Author Contributions

Qiangqiang Ouyang, Juan Wu and Miao Wu designed the system and the experiments. Ouyang implemented the system, guided the experiments, analyzed the data and wrote the article. Juan Wu reviewed the paper, contributed to the writing and to the data analysis. Miao Wu contributed to the tests with subjects, Qiangqiang Ouyang contributed to the analysis of the experiments and the materials.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Newman, D.G.; FAICD, A. An Overview of Spatial Disorientation as a Factor in Aviation Accidents and Incidents; Australian Transport Safety Bureau: Canberra, Australia, 2007.
  2. Van Erp, J.B.; Self, B.P. Tactile Displays for Orientation, Navigation and Communication in Air, Sea and Land Environments; North Atlantic Treaty Organisation, Research & Technology Organisation: Neuilly sur Seine, France, 2008. [Google Scholar]
  3. Rupert, A.H. An instrumentation solution for reducing spatial disorientation mishaps. IEEE Eng. Med. Biol. Mag. 2000, 19, 71–80. [Google Scholar] [CrossRef] [PubMed]
  4. Kim, S.H.; Kaber, D.B. Examining the Effects of Conformal Terrain Features in Advanced Head-Up Displays on Flight Performance and Pilot Situation Awareness. Hum. Factors Ergon. Manuf. Serv. Ind. 2014, 24, 386–402. [Google Scholar] [CrossRef]
  5. Rupert, A.H.; Lawson, B.D.; Basso, J.E. Tactile Situation Awareness System Recent Developments for Aviation. Proc. Hum. Factors Ergon. Soc. Ann. Meet. 2016, 60, 722–726. [Google Scholar] [CrossRef]
  6. Payette, J.; Hayward, V.; Ramstein, C.; Bergeron, D. Evaluation of a force feedback (haptic) computer pointing device in zero gravity. In Proceedings of the ASME Dynamcis Systems and Control Division; American Society of Mechanical Engineers: New York, NY, USA, 1996; Volume 58, pp. 547–553. [Google Scholar]
  7. Traylor, R.; Tan, H.Z. Development of a wearable haptic display for situation awareness in altered-gravity environment: Some initial findings. In Proceedings of the 10th Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems, Orlando, FL, USA, 24–25 March 2002; pp. 159–164. [Google Scholar]
  8. Van Erp, J.B.; van Veen, H.A. Touch down: the effect of artificial touch cues on orientation in microgravity. Neurosci. Lett. 2006, 404, 78–82. [Google Scholar] [CrossRef] [PubMed]
  9. Wickens, C.D. Processing resources and attention. In Multiple-Task Performance; CRC Press: Boca Raton, FL, USA, 1991; pp. 3–34. [Google Scholar]
  10. Wickens, C.D.; Liu, Y. Codes and modalities in multiple resources: A success and a qualification. Hum. Factors J. Hum. Factors Ergon. Soc. 1988, 30, 599–616. [Google Scholar] [CrossRef] [PubMed]
  11. Heuten, W.; Henze, N.; Boll, S.; Pielot, M. Tactile wayfinder: A non-visual support system for wayfinding. In Proceedings of the 5th Nordic Conference on Human—Computer Interaction: Building Bridges, Lund, Sweden, 20–22 October 2008; ACM: New York, NY, USA, 2008; pp. 172–181. [Google Scholar]
  12. Van Erp, J.B.; Van Veen, H.A. Vibrotactile in-vehicle navigation system. Transp. Res. Part F Traffic Psychol. Behav. 2004, 7, 247–256. [Google Scholar] [CrossRef]
  13. Ho, C.; Tan, H.Z.; Spence, C. Using spatial vibrotactile cues to direct visual attention in driving scenes. Transp. Res. Part F Traffic Psychol. Behav. 2005, 8, 397–412. [Google Scholar] [CrossRef]
  14. Carrera, A.; Alonso, A.; Rosa, R.D.L.; Abril, E. Sensing Performance of a Vibrotactile Glove for Deaf-Blind People. Appl. Sci. 2017, 7, 317. [Google Scholar] [CrossRef]
  15. Scott, J.J.; Gray, R. A Comparison of Tactile, Visual, and Auditory Warnings for Rear-End Collision Prevention in Simulated Driving. Hum. Factors J. Hum. Factors Ergon. Soc. 2008, 50, 264–275. [Google Scholar] [CrossRef] [PubMed]
  16. Morrell, J.; Wasilewski, K. Design and evaluation of a vibrotactile seat to improve spatial awareness while driving. In Proceedings of the 2010 IEEE Haptics Symposium, Waltham, MA, USA, 25–26 March 2010. [Google Scholar]
  17. Flores, G.; Kurniawan, S.; Manduchi, R.; Martinson, E.; Morales, L.M.; Sisbot, E.A. Vibrotactile Guidance for Wayfinding of Blind Walkers. IEEE Trans. Haptics 2015, 8, 306–317. [Google Scholar] [CrossRef] [PubMed]
  18. Cardin, S.; Vexo, F.; Thalmann, D. Vibro-tactile interface for enhancing piloting abilities during long term flight. J. Robot. Mechatron. 2006, 18, 381–391. [Google Scholar] [CrossRef]
  19. Kim, Y.; Harders, M.; Gassert, R. Identification of Vibrotactile Patterns Encoding Obstacle Distance Information. IEEE Trans. Haptics 2015, 8, 298–305. [Google Scholar] [CrossRef] [PubMed]
  20. McGrath, B.J.; Estrada, A.; Braithwaite, M.G.; Raj, A.K.; Rupert, A.H. Tactile Situation Awareness System Flight Demonstration; Technical Report, DTIC Document; Defense Technical Information Center: Fort Belvoir, VA, USA, 2004. [Google Scholar]
  21. Tan, H.Z.; Durlach, N.I.; Reed, C.M.; Rabinowitz, W.M. Information transmission with a multifinger tactual display. Atten. Percept. Psychophys. 1999, 61, 993–1008. [Google Scholar] [CrossRef]
  22. Tan, H.Z. A Haptic Back Display for Attentional and Directional Cueing. Haptics-e 2003, 3, 1–20. [Google Scholar]
  23. Novich, S.D.; Eagleman, D.M. Using space and time to encode vibrotactile information: toward an estimate of the skin’s achievable throughput. Exp. Brain Res. 2015, 233, 2777–2788. [Google Scholar] [CrossRef] [PubMed]
  24. Saddiki-Traki, F.; Tremblay, N.; Tremblay, R.W.; Derraz, S.; El-Khamlichi, A.; Harrisson, M. Differences between the tactile sensitivity on the anterior torso of normal individuals and those having suffered complete transection of the spinal cord. Somatosens. Mot. Res. 1999, 16, 391–401. [Google Scholar] [CrossRef] [PubMed]
  25. Weinstein, S. Intensive and Extensive Aspects of Tactile Sensitivity as a Function of Body Part, Sex and Laterality. In Proceedings of the first International Symposium on the Skin Senses, Tallahassee, FL, USA, March 1966; pp. 195–218. [Google Scholar]
  26. Van Erp, J.B. Vibrotactile spatial acuity on the torso: Effects of location and timing parameters. In Proceedings of the First Joint Eurohaptics Conference and Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems, Pisa, Italy, 18–20 March 2005; pp. 80–85. [Google Scholar]
  27. Van Erp, J.B. Presenting directions with a vibrotactile torso display. Ergonomics 2005, 48, 302–313. [Google Scholar] [CrossRef] [PubMed]
  28. Van Erp, J.B.; Van Veen, H.A.; Jansen, C.; Dobbins, T. Waypoint navigation with a vibrotactile waist belt. ACM Trans. Appl. Percept. 2005, 2, 106–117. [Google Scholar] [CrossRef]
  29. Alluisi, E.A.; Warm, J.S. Things that go together: A review of stimulus-response compatibility and related effects. Adv. Psychol. 1990, 65, 3–30. [Google Scholar]
  30. Cholewiak, R.W.; Collins, A.A. The generation of vibrotactile patterns on a linear array: Influences of body site, time, and presentation mode. Percept. Psychophys. 2000, 62, 1220–1235. [Google Scholar] [CrossRef] [PubMed]
  31. Na, L. Optimization of Vibrotactile Device Experimental Study on Perception Characteristics. Ph.D. Thesis, Southeast University, Nanjing, China, 2016. [Google Scholar]
  32. Sachs, R.M.; Miller, J.D.; Grant, K.W. Perceived magnitude of multiple electrocutaneous pulses. Atten. Percept. Psychophys. 1980, 28, 255–262. [Google Scholar] [CrossRef]
  33. Kirman, J.H. Tactile apparent movement: The effects of interstimulus onset interval and stimulus duration. Atten. Percept. Psychophys. 1974, 15, 1–6. [Google Scholar] [CrossRef]
  34. Hayward, V. A brief taxonomy of tactile illusions and demonstrations that can be done in a hardware store. Brain Res. Bull. 2008, 75, 742–752. [Google Scholar] [CrossRef] [PubMed]
  35. Lederman, S.J.; Jones, L.A. Tactile and Haptic Illusions. IEEE Trans. Haptics 2011, 4, 273–294. [Google Scholar] [CrossRef] [PubMed]
  36. Wolpaw, J.R.; Ramoser, H.; McFarland, D.J.; Pfurtscheller, G. EEG-based communication: improved accuracy by response verification. IEEE Trans. Rehabil. Eng. 1998, 6, 326–333. [Google Scholar] [CrossRef] [PubMed]
  37. Wolpaw, J.R.; Birbaumer, N.; McFarland, D.J.; Pfurtscheller, G.; Vaughan, T.M. Brain-computer interfaces for communication and control. Clin. Neurophysiol. 2002, 113, 767–791. [Google Scholar] [CrossRef]
  38. Gescheider, G. The Classical Psychophysical Methods. In Psychophysics: The Fundamentals, 3rd ed.; Lawrence Erlbaum Associates: Mahwah, NJ, USA, 1997; Chapter 3. [Google Scholar]
  39. Eriksson, L.; van Erp, J.; Carlander, O.; Levin, B.; van Veen, H.; Veltman, H. Vibrotactile and visual threat cueing with high G threat intercept in dynamic flight simulation. Proc. Hum. Factors Ergon. Soc. Ann. Meet. 2006, 50, 1547–1551. [Google Scholar] [CrossRef]
  40. Hoh, R.H.; Mitchell, D.G. Handling-qualities specification-a functional requirement for the flight control system. In Advances in Aircraft Flight Control; CRC Press: Boca Raton, FL, USA, 1996; pp. 3–33. [Google Scholar]
Applsci 07 01291 g001 550
Figure 1. Vibrotactile actuator.
Figure 1. Vibrotactile actuator.
Applsci 07 01291 g001
Applsci 07 01291 g002 550
Figure 2. The arrangement of distribution of tactors around the torso and prototype of hlthe designed vibrotactile vest. In frontal of body, five tactors are located on the lower abdomen, upper abdomen, slightly below the chest, middle of chest and below of the clavicle respectively. In back of body, five tactors are located on the hip above, central spine, right shoulder, left shoulder, and cervical spine respectively. In the left or right side of body, five tactors, from down to up, are located on hipbone above, rib, oxter, chest and shoulder respectively.
Figure 2. The arrangement of distribution of tactors around the torso and prototype of hlthe designed vibrotactile vest. In frontal of body, five tactors are located on the lower abdomen, upper abdomen, slightly below the chest, middle of chest and below of the clavicle respectively. In back of body, five tactors are located on the hip above, central spine, right shoulder, left shoulder, and cervical spine respectively. In the left or right side of body, five tactors, from down to up, are located on hipbone above, rib, oxter, chest and shoulder respectively.
Applsci 07 01291 g002
Applsci 07 01291 g003 550
Figure 3. Overview of vibrotactile system for cueing flight attitude. (a) System block diagram of vibrotactile TSAS and Tactor arrangement around the torso of pilot. The controller receive flight attitude information from the geomagnetic sensor and activate vibrotactile vest to vibrate in the pattern corresponding to the attitude; (b) Schematic diagram of encoding flight states for different tactor columns; (c) Definition of Flight attitude information of aircraft.
Figure 3. Overview of vibrotactile system for cueing flight attitude. (a) System block diagram of vibrotactile TSAS and Tactor arrangement around the torso of pilot. The controller receive flight attitude information from the geomagnetic sensor and activate vibrotactile vest to vibrate in the pattern corresponding to the attitude; (b) Schematic diagram of encoding flight states for different tactor columns; (c) Definition of Flight attitude information of aircraft.
Applsci 07 01291 g003
Applsci 07 01291 g004 550
Figure 4. Schematic diagram of tactile apparent motion.
Figure 4. Schematic diagram of tactile apparent motion.
Applsci 07 01291 g004
Applsci 07 01291 g005 550
Figure 5. A subject during the experiment.
Figure 5. A subject during the experiment.
Applsci 07 01291 g005
Applsci 07 01291 g006 550
Figure 6. Schematic diagram of coding methods with combination of two parameters: (a) location and intensity (LI); (b) location and rhythm(LR); (c) location andmode(LM). For the convenience of labelling angel intervals, five degrees in vertical coordinate indicates interval [5, 10) in Table 3, 10 indicates [10, 15), and so forth. The cueing time is maximum time to convey a vibrotactile pattern. Different colour blocks represent tactors locating at different sites in a column.
Figure 6. Schematic diagram of coding methods with combination of two parameters: (a) location and intensity (LI); (b) location and rhythm(LR); (c) location andmode(LM). For the convenience of labelling angel intervals, five degrees in vertical coordinate indicates interval [5, 10) in Table 3, 10 indicates [10, 15), and so forth. The cueing time is maximum time to convey a vibrotactile pattern. Different colour blocks represent tactors locating at different sites in a column.
Applsci 07 01291 g006
Applsci 07 01291 g007 550
Figure 7. Recognition accuracy and reaction time (s) for each coding method in Experiment 1. Error bars indicate standard error.
Figure 7. Recognition accuracy and reaction time (s) for each coding method in Experiment 1. Error bars indicate standard error.
Applsci 07 01291 g007
Applsci 07 01291 g008 550
Figure 8. Schematic diagram of coding methods with combination of three parameters: (a) location, rhythm of times and intensity (LRI1); (b) location, rhythm of duration and intensity (LRI2); (c) location, rhythm of times and intensity (LRI3); (d) location, rhythm and mode (LRM). For the convenience of labelling angel intervals, five degrees in vertical coordinate indicates the interval [5, 10) in Table 3, 10 indicates [10, 15) and so forth.
Figure 8. Schematic diagram of coding methods with combination of three parameters: (a) location, rhythm of times and intensity (LRI1); (b) location, rhythm of duration and intensity (LRI2); (c) location, rhythm of times and intensity (LRI3); (d) location, rhythm and mode (LRM). For the convenience of labelling angel intervals, five degrees in vertical coordinate indicates the interval [5, 10) in Table 3, 10 indicates [10, 15) and so forth.
Applsci 07 01291 g008
Applsci 07 01291 g009 550
Figure 9. Recognition accuracy and reaction time (s) for each coding method in experiment 2. Error bars indicate standard error.
Figure 9. Recognition accuracy and reaction time (s) for each coding method in experiment 2. Error bars indicate standard error.
Applsci 07 01291 g009
Applsci 07 01291 g010 550
Figure 10. Information transfer rate for different coding methods; the error bars indicate standard error.
Figure 10. Information transfer rate for different coding methods; the error bars indicate standard error.
Applsci 07 01291 g010
Applsci 07 01291 g011 550
Figure 11. The curve of recognition accuracy change with set angle in preferred coding method. Error bars indicate standard error.
Figure 11. The curve of recognition accuracy change with set angle in preferred coding method. Error bars indicate standard error.
Applsci 07 01291 g011
Table
Table 1. Division of angle range in each flight state.
Table 1. Division of angle range in each flight state.
GroupRanges of Angle (Degree)Flight Status
≥90Emergency
5[70, 80) [80, 90)Acrobatic flight
4[50, 60) [60, 70)
3[35, 40) [40, 45) [45, 50)
2[20, 25) [25, 30) [30, 35)Normal flight
1[05, 10) [10, 15) [15,20)
[0, 5)Steady flight
Table
Table 2. Coding parameters for tactors.
Table 2. Coding parameters for tactors.
Parameters for Single TactorVibrating location (P), Intensity (I), Rhythm (R)
Parameters for Multiple TactorsVibrating mode (M)
Table
Table 3. Post hoc test for different combinations of coding parameters with LSD correction.
Table 3. Post hoc test for different combinations of coding parameters with LSD correction.
CM(I)CM(J)Mean Difference/I-J(Z)Sig
LILR−0.3580 *0.000
LILM−0.2420 *0.005
LRLM0.1160 *0.122
* represents the mean difference is significant at the 0.05 level.
Table
Table 4. Post hoc Test for different combinations of coding parameters with LSD correction.
Table 4. Post hoc Test for different combinations of coding parameters with LSD correction.
CM(I)CM(J)Mean Difference/I-J(Z)Sig
LRLRI1−0.17367 *0.004
LRLRI2−0.006100.916
LRLRI3−0.11890 *0.043
LRI1LRI20.17976 *0.000
LRI1LRI30.054760.219
LRI2LRI3−0.12500 *0.007
PRMPRI10.014240.694
* represents the mean difference is significant at the 0.05 level.
Table
Table 5. MANOVA in the preferred coding method.
Table 5. MANOVA in the preferred coding method.
SourceQuadratic SumDofMean SquareFSig
States0.01930.0060.1970.898
Locations0.28130.0942.9700.032
Rhythms0.10920.0541.7210.181
States × Locations0.32390.0361.1360.337
States × Rhythms0.12160.0200.6380.700
Locations × Rhythms0.04940.0120.3850.819
States × Locations × Rhythms0.292120.0240.7700.681
Table
Table 6. Performance comparison between our work and previous TSASs.
Table 6. Performance comparison between our work and previous TSASs.
TSASsTactor ArrangementNumber of Used TactorsCoding MethodsResolution /Degree
This work4 × 5 matrix vest20Coding with LRM5
[3]8 × 5 matrix vest40Coding with LI5
[18]8-tactor belt8Coding with LSimple information
[39]60-tactor jacket60Coding with L30

Share and Cite

MDPI and ACS Style

Ouyang, Q.; Wu, J.; Wu, M. Vibrotactile Display of Flight Attitude with Combination of Multiple Coding Parameters. Appl. Sci. 2017, 7, 1291. https://doi.org/10.3390/app7121291

AMA Style

Ouyang Q, Wu J, Wu M. Vibrotactile Display of Flight Attitude with Combination of Multiple Coding Parameters. Applied Sciences. 2017; 7(12):1291. https://doi.org/10.3390/app7121291

Chicago/Turabian Style

Ouyang, Qiangqiang, Juan Wu, and Miao Wu. 2017. "Vibrotactile Display of Flight Attitude with Combination of Multiple Coding Parameters" Applied Sciences 7, no. 12: 1291. https://doi.org/10.3390/app7121291

APA Style

Ouyang, Q., Wu, J., & Wu, M. (2017). Vibrotactile Display of Flight Attitude with Combination of Multiple Coding Parameters. Applied Sciences, 7(12), 1291. https://doi.org/10.3390/app7121291

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop