INTERACTION SILENT SPEECH INTERFACES × × SSI capture low amplitude sounds generated by a tissue-conducted vibration of vocal-tract resonance of laryngeal airflow noise. × × Commands can be softly whispered without concerning the existence and implications of the background noise or the other people around The vocalized speech can be distinguished from murmurs for giving commands during a voice-call. Thus, user may still use SSI to give commands (e.g. terminate call) even when the voice call is initiated. In other words, SSI identify pronounced phonemes when intelligible acoustic signal is unavailable e.g. nonaudible murmurs. × RECOGNITION Murmurs can be accurately recognized even in noisy environments and interpreted as different commands for interaction with mobile device. × Commands are small set of pre-defined single-words, which are optionally followed by parameters (e.g. Call Elliot). × Commands and parameters can accurately be recognized as they consist of a small set of keywords, numbers and additional terms such as the names in the address book. × Commands or interrupts can also be used for non-communication related tasks such as checking the signal strength or battery level via auditory feedback. CONCEPT USAGE 02 01 FUNCTIONALITIES × The proposed mobile phone is designed using low-cost and commercially available hardware components while focusing on the essence of mobile communication and connectivity. × However, it supports lots of other non-visual functionalities such as sound recording, playing music, push-to-talk, clock, alarm, calendar, calculator, etc. DESIGNING MOBILE PHONES USING SILENT SPEECH INPUT AND AUDITORY FEEDBACK HARDWARE 03 KAMER ALI YUKSEL SINAN BUYUKBAS SERDAR HASAN ADALI COMPUTER VISION AND PATTERN ANALYSIS LABORATORY (VPALAB) VISUAL COMMUNICATION DESIGN (VAVCD) COMPUTER GRAPHICS LABORATORY SABANCI UNIVERSITY, ORHANLI -TUZLA, 34956 ISTANBUL SABANCI UNIVERSITY, ORHANLI -TUZLA, 34956 ISTANBUL SABANCI UNIVERSITY, ORHANLI -TUZLA, 34956 ISTANBUL KAMER@SABANCIUNIV.EDU SINANBUYUKBAS@SABANCIUNIV.EDU SERDARADALI@SABANCIUNIV.EDU MOTIVATION OUR SOLUTION × New features are continuously × Most of those features are added to mobile phones due to their competitive market. × Voice call and text messaging are fundamental features among others e.g. MMS, e-mail, Internet, music, gaming and photography. not indispensable for mobile communication or connectivity and they are not utilized by the majority (86%) of users. × Additional features increase the complexity and cost of mobile devices and their network infrastructure. × Novel design for a basic mobile phone considering only fundamental communication features to make it more accessible concerning interaction complexity and cost. × Reducing the mobile phone to a purely voice-driven device by combining audible voice for human-to-human communication with silent speech for human-computer interaction × The proposed device is in the form of a headset and takes input through a SSI and gives auditory feedback through an earphone as response to their commands or as interrupts. HARDWARE × 3 non-invasive and low-cost technologies are available to build SSI systems. Throat microphone: captures speech signals from both sides of the Adam’s apple In-ear microphone: inserted into the ear canal for the same purpose. Non-audible murmur (NAM) microphone: consists of a silicon conductor and a condenser microphone and placed on the neck below ear. conductive signals. body × Cheap and commercially available in compact forms. × Allows recognizing non-audible whisper with high accuracies. × The in-ear microphone or NAM microphone can be utilized to have a more compact device due to the proximity of their targeted area to the ear. × The throat microphone may be preferred due to availability and cost or just for visual or ergonomic design purposes. ADVANTAGES × SSI preserve the advantages of conventional speech-based voice control interfaces. × They can be utilized in noisy environments or when the silence is required due to privacy or social acceptance, even by speechhandicapped. 04 EXTENSION × Resistant to ambient background noise because they are based on × More complex interactions are possible through external peripherals or embedded extensions to the low-cost device. × For example, Pico-projector can be used to show visual feedback on walls or hands of users or Camera can be used to capture photographs or stream videos their viewpoint. × Motion sensors (e.g. accelerometer, gyroscope) can be embedded to the throat microphone itself in order to perform gestures when necessary.