CA2776856A1 - Voice recognition motion sensing touch free toilet seat raising and lowering assembly - Google Patents

Description

TITLE: VOICE RECOGNITION RESPONSIVE MOTION SENSING TOUCH FREE
TOILET SEAT RAISING AND LOWERING ASSEMBLY V
BACK GROUND---FIELD OF THE EMBODOIMENTS
These embodiments relate to the technical field of toilet seat lifting devices, and speech recognition responsive and motion sensing devices. The embodiments relate to toilet seats, and toilet seat lids.
BACK GROUND OF THE EMBODIMENTS
Toilet seats can become contaminated by germs, which can be transmitted to a person's hands, during contact with the seat. Hands may then spread the bacteria and viruses to the person's mouth, to other surfaces, to other people, and to food, thereby increasing the risk of food poisoning, and transfer of diseases. Many people don't like to touch a seat or lid, that has urinary or fecal matter on them. Some people try to use their foot or shoe, to raise or lower the toilet seat, but this has drawbacks, such as the possibility of losing one's balance and falling, it also requires a certain amount of skill, to stand on one leg, and use the other leg to position the seat.
The advantages of the a voice recognition and motion sensing input device, toilet seat raising and lowering system, compilation, assembly, device, apparatus include, without limitation, are an improved toilet seat that has, a voice recognition system that allows the user, operator, consumer, to move a toilet seat and lid, to a desired position, by spoken words. The motion sensing device allows a user to move the toilet seat and lid using hand motions. A user moves their hands in front of the assembly, within the webcam view frame A user can raise or lower the toilet seat or lid without having to touch them .This reduces hand contact with the seat and lid. Bacteria and viruses on the seat and lid are unable to attach to a user's hands, without hand contact with the seat The transmission of bacteria and viruses, that are on the seat and lid, to the hands, is substantially reduced. The system lowers the amount of bacteria that is transmitted from the toilet seat to hands, which lowers the amount of bacteria that is transmitted to the user, such as, bacteria from hands to mouth. By lowering the quantity of bacteria on hands, limits the amount of bacteria traveling from a user's hands to food.
The amount of Bacterial transmission from hands to surfaces, and transmission from hands to other people, is limited. Lowering the cross contamination of bacteria and viruses, from toilet seat to users hands, lessens peoples exposure to food poisoning, and disease pathogens.
Several types of hands free toilet seat raising and lowering devices have been proposed, for example:
Baumoel in, U.S. Pat. No. 7917973 (2011) shows an infrared hands free system, for lifting and lowering a toilet seat.
Mechanical force devices, such as, Borochov and Gitelis, U.S. Pat. No. 8020221 (2011) uses a pneumatic toilet seat lifting device, which uses foot force to activate the toilet seat lifting device. The draw backs, with such a system, are a user needs to physically find the foot lever, and apply force with their foot.
Veals, in U.S. Pat. No. 5307524 uses an electric switch to activate, the raising and lowering of a toilet seat, one disadvantage, is the user needs to locate the switch with their foot.
More toilet seat lifting devices are shown in the following patents.
Title: Hands Free Toilet, Patent number CA 2624181, issue date 2011-11-22, assignee Doucet, Sheryl Title: Automatic Toilet Seat Lifting and Lowering Device, Patent number US
7293297, issue date 2007-11-13, assignee Ryosuike, Hayashi Title: Automated Toilet Seat, Publication number US 2009/0106885 Al, filing date 2007-10-26, assignee Sagre, Miguel Tile: Toilet Seat Lifting and Lowering Device, Patent number CA 2585807, issue date 2011-0201, assignee, Pantos, William P.
Title: Powered Toilet Seat Positioner, Patent number US 6915532, issue date 2005-07-12, assignee Mohammed , Gaffar Title : Sanitary Toilet Seat, Publication number US 2011/0191950 Al, filling date 2010-02-11, assignee Liu, Lin-Ho The Bottoms Up company , Waslet company, itouchless company, sell infrared hands free toilet seat, lifting and lowering systems.
A disadvantage with infrared systems is that physical movements are needed, to control the system. Infrared systems may be inadvertently activated if a user gets too close to the infrared sensor. Though such toilet seat lifting devices have achieved considerable popularity, and commercial success, there has been a continuing need for improvement.
In conclusion, insofar as I am aware, no toilet seat formerly developed provides a movement and contact free, spoken word voice recognition responsive, toilet seat lifting apparatus. Spoken words within 3 to 5 meters activate the raising or lowering of the toilet seat and lid. Defects of other devices, is that, physical presence movement is required near a sensor to operate them. Infrared sensor systems may activate movement of the toilet seat inadvertently. The motion sensing device responds to a user's movements, to raise and lower the toilet seat and lid. The device operates at variable distances to the user. The motion sensing devices body gesture recognition software is less likely too inadvertently active the seat or lids motion, then the infrared sensors.
Accordingly several advantages are to provide an improved hands free toilet seat, as a means of providing a more sanitary, fun, and healthy experience, for a person. Still further advantages will become apparent from a study of the following description and the accompanying drawings.
SUMMARY
Various embodiments of the speech recognition toilet seat assembly enable people to raise and lower a toilet seat with verbal input. Incorporating facial recognition in the system, and having a user who is in the view of the assembly video cam, enables the system to move the seat to the user's desired seat position. Using motion sensing enables a user to intuitively use hand motions to move seat the position, without touching the seat The capabilities and functions of the seat assembly, bring the future of sanitary bathroom practices and advances, to the present now.
DRAWINGS¨Figures Fig 1 shows a perspective view the voice recognition seat assembly attached to a toilet.
Figs 2A and 2B show the seat assembly attached to a toilet, and shows the voice recognition microphone, right perspective view and left perspective view.
Figs 3A and 3B shows the exposed inside of the assembly housing, right and left view.
Figs 4A and FIG. 5B, shows the longitudinal adjustable toilet assembly attachment plate.
Figs 5A and 58 shows the seat assembly with the facial recognition video cam, outside front view, and exposed inside housing front Fig 6A and 6B shows seat and lid attached to toilet, view of lid up.
Figs 7A and FIG.7B shows seat and lid, exposed inside view of the front housing, and inside view of the inside of the housing back.
Figs 8A and 86 show a one way water drain, at the base of the seat assembly.
Figs 9A and 9B shows right front view of seat and lid assembly, and exposed back inside of the housing view, with ACDC adapter plugged in.
Fig 10 shows a flow chart of the connection between assembly components.
Fig 11 shows the connection and interconnects, between operations of the components.
Figs 12A and 12 shows a one shaft toilet seat and a one shaft lid, with lid in the up position, in accordance with another embodiment Figs 13A and 136 shows a cross section of the interchangeable seat compressible shaft, and compressible shaft spring.
Figs 14A and 146 shows the one shaft connection interchangeable toilet seat with screw on support securing caps, outside and inside exploded view, in accordance with another embodiment.
Fig 15A shows the ACDC adapter, in battery charging fashion.
Fig 15B shows the toilet seat assembly, with the facial recognition video cam widow.
Figs 16A and 166 show the front interchangeable toilet seat. Attached and detached form the housing base, in accordance with another embodiment.
Figs 17A and 176 shows the interchangeable seat, exploded view inside the housing, the right side and left side, in accordance with another embodiment.

Figs 18A and 18B show the width adjustable attaching plate unsecured, and secured to the assembly.
Fig 19 shows the toilet seat in the up position, attached to a toilet.
Fig 20A front view of Kinect seat and lid assembly on toilet.
Fig 20B close up front view of Kinect seat and lid assembly, on toilet.
Fig 21A and Fig 21B, view inside housing of Kinect seat and lid assembly.
Fig 22A view Kinect seat assembly, on toilet.
Fig 22B view inside Kinect assembly housing, on toilet.
Fig 23A side view Kinect assembly seat.
Fig 24 back view inside housing of Kinect seat assembly.
Fig 25 back inside view housing of Kinect seat and lid assembly.
Fig 26 perspective view housing with holographic projector Kinect assembly Fig 27 is a diagram of the sensory language microcontroller Fig 28 is a diagram of the sensory language microcontroller Fig 29 diagram speech recognizer REFERENCE NUMERALS
40 toilet seat 41 seat, web cam assembly 42 toilet lid or cover 43 toilet lid or cover seat and lid 44 housing front 46 housing back 47 seat and lid assembly 48 base housing 49 base one support connection seat 50 housing left side 51 left side of the housing seat and lid assembly 52 housing right side 53 housing right side seat and lid assembly 54 microphone seat and lid assembly 55 speaker seat and lid assembly 56 video camera or video cam window 57 microcontroller web cam assembly 58 web cam 60 microphone 61 web cam seat and lid assembly 62 speaker 63 system on off button switch 64 battery lithium ion 65 low battery light indicator seat and lid assembly 66 battery release button 67 reset button seat and lid assembly 68 low battery light indicator 69 reset button system micro controller 70 reset button web cam assembly 71 alternating current to direct current plug 72 alternating current direct current adapter 73 secure digital card reader 74 nut 75 water proof flap secure digital card reader 76 bolt 77 secure digital card reader Kinect computer assembly 78 rotating shaft 79 rotating shaft seat and lid 80 shaft o ring 81 shaft o ring seat and lid assembly 82 ball bearings 83 ball bearings seat and lid assembly 84 washer rotating shaft 85 washer rotating shaft seat and lid assembly 86 servo motor 87 servo motor Kinect, seat and lid 88 rotation sensor 89 servo motor seat and lid assembly 90 rotating circuit 92 wire electrical 94 drain one way 95 servo motor controller web cam assembly 96 servo motor controller 97 servo motor controller Kinect sensor assembly 98 computer 99 servo motor controller seat and lid assembly 100 ultra violet light emitting diodes 102 internet Ethernet connection outlet 103 internet Ethernet connection outlet Kinect assembly 104 Wi-Fl wireless internet card 105 Wi-Fi wireless internet card Kinect assembly 110 interchangeable seat 114 liquid crystal display 116 control button display menu 118 slip clutch 120 microcontroller microphone assembly 121 microcontroller web cam assembly 122 shaft washer support 124 toilet bowel attachment plate 126 ultrasonic sensor assembly 127 toilet bowel 128 toilet bowel seat and lid 130 plate assembly lock 132 one support toilet seat 134 one support lid 135 shaft with screw treads for screw on for caps 136 screw on cap holder for one support seat and lid 138 compressible shaft 140 compressible shaft spring 142 toilet seat interchangeable 144 web cam assembly 146 quick release button interchangeable toilet seat 148 interchangeable toilet seat base assembly 150 locking shaft anchor interchangeable toilet seat 152 width adjustable plate, assembly toilet coupling 154 anti-bacterial coating 155 Kinect motion sensing, voice recognition and facial recognition device 156 computer, laptop computer, XBOX console, net book 157 microcontroller Kinect assembly seat assembly 158 Kinect, seat and lid assembly 159 microcontroller Kinect 160 seat, Kinect assembly 162 lid, Kinect assembly 164 speaker Kinect assembly 168 slip clutch, seat and lid assembly 170 web cam, seat and lid assembly 172 battery, seat and lid assembly 174 housing top, seat and lid assembly 176 battery release button, seat and lid assembly 180 reset button computer Kinect assembly 182 battery, web cam assembly 184 shaft o ring, seat and lid 186 base, web cam 188 housing front, Kinect microcontroller 190 housing front Kinect computer 192 toilet, Kinect computer 194 front housing, web cam 195 holographic projector 196 housing front seat and lid DETAILED DESCRIPTION- FIRST EMBODIMENT
One embodiment, of a toilet seat assembly, system, apparatus is illustrated in Fig 1, Fig 2A (right side view), 2B and1B (left side view). A housing assembly baying a base 48, front 44, left side 50, right side 52 and back 46, where the parts of the housing components meet, are rubber gaskets, silicon caulking, o rings 80, or other water sealant materials, used to prevent water from entering the housing. Many components of the assembly are made of high strength water resistant plastic, and other materials.
Some of components of the assembly are secured to each other, by plastic screws and stainless steel zinc coated screws and nuts.
A toilet seat 40 is connected to two rotating shafts 78. From the toilet seat the shafts goes through washers 84 that are between the seat and housing. One shaft 78 enters the housing through the right side 52. The shaft travels through a lubricated o ring 80.
The length of the shaft 78 then goes through, two plastic support ball bearings 82 that are mounted to the housing. The shaft 78 connects to a rotation sensor 88, which is mounted to the housing. The rotation sensor 88, is connected to, a microcontroller, microcomputer 120. The microcontroller 120 is an integrated circuit made by Sensory Company, Sensory's NLP-5 x Natural Language Processor 120. The microcontroller 120 is programmed, with Sensory's TrulyHandFree voice technology, speech recognition, voice recognition software. (Fig 76) The user can chose a male or female voice for the microcontrollers 120 synthesized talking voice. The system can converse in eight or more languages, Korean, French, Mandarin, German, Spanish, Italian, English and Japanese. Programming of the assembly, is accomplished, by following programming manuals that accompany Sensory's TrulyHandsFree speech software. A

water resistant microphone 62 is connected to the microcontroller 120. The left side shaft, enters the left side assembly wall 52, going through the plastic washer 84, the o ring 80, two ball bearings 82, the end of the shaft 78 connects, to a servo motor 86. The servo motor 86 connects to, a servo controller 96 and the microcontroller 120 (Fig 3A).
A system reset button 69, connects to the microcontroller 120. The servo controller 96 connects to a rechargeable lithium ion battery 64, and an alternating current to direct current adapter 72. Fig 2B shows a system on off button 63, which connects to the servo motor controller 96.
Operation First Embodiment The microphone 60 listens for a user's vocalised commands (Figs 2A and 2B). In listening mode the system uses a minimal amount of power. The system activates when hearing a user command. A user can verbally direct the seat 40 to move up or down to a resting position. Saying the word up, moves the seat 40 up, saying the word down, moves the seat 40 down. The seat 40 rests on the toilet bowel rim 127 when it's in the down position, when the seat 40 is in up position the seat rests against the toilet water tank 127, bathroom wall, or the seat assembly. Spoken words are sensed by the microphone 60, which sends signals to the microcontroller 120, the words are analysed by the microcontrollers voice recognition software 120. The words meanings and intent are analysed, the users desires are understood, the right system information is found, the system responds with the desired function. The system matches the spoken words to a system response, such as, raising or lowering the seat.40 The system is connected to a reset button 69, and can be reset by pushing the system reset button 69, illustrated in Fig 2A .Pressing the reset button 69, will turn off, then turn on the system, including the microcontroller 120, or computer 98.
Power Supply The battery connects to the motor controller and the alternating current to direct current adapter 72, abbreviation ACDC adapter 72. The, ACDC adapter 72 has a three pronged grounded plug, and plugs into a grounded electrical supply outlet The ACDC
adapter 72 connects by electrical wire to the assembly by plugging in to the assembly's ACDC plug, shown in 9A and 98 The ACDC adapter 72 incorporates a resettable circuit breaker, and circuit breaker test switch. A low battery light indicator 68 connects to the microcontroller 120, and motor controller 96. Fig 15A shows the ACDC adapter 72, changing the battery 64 outside of the assembly. The battery 64 can also be changed while in the assembly Attaching The Assembly To TheToilet Description Figs 18A and 18B shows the base 48 of the housing attached to a plate152 that is attachable to a toilet 127. The toilet plate 152 attaches to the toilet 128, by two bolts 76 and two nuts 74. The plate 152 can be adjustable for the two toilet holes (not shown), width distance from each hole. The housing assembly slides on to the plate 152, and locks on to the plate by moving a sliding latch 130, into locked position.
Figs 4A and 4B
shows a longitudinal adjustable plate, which the assembly attaches to.
Attaching The Assembly To The Toilet Operation The assembly attaches to a plate152 that attaches to the toilet 127. Figs 18A
and18B shows a plastic plate 152, that allows for two bolts 76 and nuts 74, that can slide width wise to adjust for differing toilet bowel hole distances, the plate attaches to the toilet 127. Figs 4A and 4B shows a plate 124 that allows the two bolts to slide longitudinally. The assembly slides on to the anchor plate. Figs 4A, 4B, 18A
and 18B
show how the plate is secured to the assembly by sliding a latch down 130, thus securing the assembly to the toilet 12. Other seat assembly and toilet connections include Fig 19 which show the seat assembly permanently attached to the toilet, the seat assembly and toilet being one assembly. Fig 6B shows the seat and lid seat assembly permanently attached to a toilet, the seat and lid assembly, and toilet being one assembly.
The housing has a one way water drain 94 (Fig 8B), located at the base. Any possible moisture is directed to the outside of the housing. The housing is water resistant and water proofed.
ALTERNATE EMBODIMENTS
Facial Recognition Description An alternate embodiment, specification, uses a microcontroller 121 that is programmed with facial recognition software that is made by, Luxand Company.
The microcontroller 121 incorporates a microprocessor, central processing unit with a speed of 1.66 megahertz. A video camera, video cam, web cam 58 connects to the microcontroller 121. Luxand facial recognition software 121 is programmed using Luxand programming manuals. The camera 58 looks out of the front of the housing 194, with an unobstructed view, whether the seat 41 is positioned up or down. The system is connected to a reset button 70, and can be reset by pushing the system reset button 70, Figs 5 A and 5B.
Facial Recognition Operation The assembly incorporates a facial recognition system 144. The system may use a computer (not shown) or microcontroller, 121 which are programmed with facial recognition software, as illustrated in Figs 3A, 3B and 9A. The facial recognition and speech recognition software enables the system to identify, recognize a user, and associate the user to their system favourites. The facial recognitions system visually identifies a user. When the user is identified the system moves the seat 41 to a position, which is the users, usual desired position. The system remembers a user's preferences, from their initial use of the system. User preferences or settings may include seat 41 position, sound effects or music, desired seat 41 positions when vacating the assembly.
The facial recognition software is based on the ability to recognize a face and measure the various features on the face. Two dimensions, 2D, or three dimension, 3D, facial recognition software can be used, to identify a user. A video camera 58 or a web cam 58, positioned in the front of the housing 194, viewing through a transparent window 56, permits an unobstructed view of a user, whether the seat 41 is up or down. The video cam 58 is connected to the microcontroller 121 and supplies images of users to the facial recognition software programmed in the microcontroller 121. A user only needs to be within view of the video cam 58 to activate the facial recognition software. The distance at which facial recognition software activates the system, and moves the seat and lid to the user's usual positions, is adjustable by the user.
Motion Sensing Kinect Sensor Device Description An alternate embodiment incorporates a Kinect motion sensing, voice recognition and facial recognition device 155 made by the Microsoft Company. In one embodiment the Kinect connects to a microcontroller 159, shown in fig 22A, fig 22B, fig 23, and fig 24. In another embodiment the Kinect is connected to a computer 156. The computer can be XBOX 360 console 156 made by Microsoft Company. The computer 156 may be a Toshiba laptop computer 156 made by the Toshiba company, with a Intel i3 CORE
processor, made by the Intel company, or a net book computer 156, made by a variety of different computer companies, as shown if fig 20, fig 21, fig 25. The computer 156 is programmed using, Kinect for Windows software developer's kit, SDK 156. The microcontroller 159 is programed using, Kinect for Windows software developer's kit, SDK 156 The assembly connects to the computer 156. The computer 156 is inside the housing 158. The computer connects to the servo controller 97. The computer connects to the servo motors 87. The computer 156 or microcontroller 159 connects to a reset button 180. The computer 156 connects to the Internet using a Ethernet cable plug outlet 103, or wirelessly using a WI-Fl card 105. The computer connects to a touch sensitive holographic projector 195. The Kinect sensors 155 look out of the front of the housing 190, with an unobstructed view of the user area in front of the housing 190 and toilet 192.
Kinect Motion Sensing Kinect Sensor Device Operation An additional embodiment uses the Kinect motion sensor 155 connected to a computer, XBOX console 156, or to a microcontroller 159 to position the seat 160 and lid 162 ,shown if fig 20, fig 21, fig 22A, fig 22B, fig 23, fig 24, and fig 25. The Kinect system 155 uses facial recognition software to identify a user. The user waves a hand in front and within view of the Kinect microcontroller assembly 157, or the Kinect computer assembly 158 to activate the Kinect sensor 155. The Kinect system uses skeletal mapping to analyze body gestures, for movement and position. The gestures are associated to, and activate system functions. The Kinect 155 enables the user to control and interact with the system without the need to touch the seat 160, to raise or lower it.
To raise or lower the seat 160, the user simulates raising or lowering the seat 160 with their hand. The assembly's seat 160 follows the hand motion of the user raising and lowering the seat 160. To move the lid 162 the users simulates the raising and lowering of the lid 162 with their hand. The seat 160 and lid 162 move separately. If the lid 162 is resting on the seat 160 the hand simulating raising, will raise the lid 162 first, then a further simulated raising with the hand will raising the seat 160. If both seat 160 and lid 162 are in the up position, the hand simulating lowering with lower the lid 162 and a further simulated lowering will lower the seat 160. The seat 160 and lid 162 are programmed to move one at a time, to avoid moving in a direction against each other, ex. lid moving down with the seat moving up. The Kinect 155 utilizes a natural user interface for sensing body and hand gestures and spoken commands. Vocal commands can be used to raise and lower the seat 160 and lid 162, using the systems voice recognition software 162. Vocal commands or body and hand gestures can be used, to control the systems functions such as, music and sound effects, etc. The user can access the internet with vocal commands, through the microphones 155 the retrieved information is broadcast audibly to the user by the speaker 164. The Kinect 155 uses four microphones155 to recognize and separates a user's voice from other noises in the room. The user may verbally chat with other users, using the internet connection. The device creates a digital skeleton of a user, based on depth data that tracks the entire body. The device uses a red, green, blue light detecting camera, RGB camera 155, depth sensor and multi array microphone 155 running proprietary software, which provide full body three dimensional motion captures, facial recognition, gesture recognition and voice recognition capabilities. The depth sensor 155 consists of an infrared laser projector combined with a monochrome complementary metal oxide semiconductor CMOS sensor 155, which captures video data in three dimensions 3D, under any ambient light conditions. The motion sensor can be used at night in darkness, without light, or in lighted environments. Speech recognition is supported in the following languages, Australia, English, Canada, English, New Zealand, English Ireland, English United Kingdom, English United States, Canada French, France French, Mexico Spanish, Spain Spanish, German, Italian and Japanese. The Kinect 155 which is connected to the computer 156, is programmed using, the Microsoft windows commercial software development kit, SDK. The Kinect 155 which is connected to the microcontroller 156, is programmed using, the Microsoft windows commercial software development kit, SDK. The SDK kit builds toilet seat 160 positioning and assembly function applications with programming languages C++, C#, and Microsoft Visual Studio Ultimate, FAAST key mapping software.
The Kinect 155 is connected to the computer 156, using the FAAST key mapping software is one of many ways to program the motion sensing system, show in fig 27.
Key mapping associates body gestures to the activation, pressing of a key on a keyboard 156. The key activation is associated with the activation of a system function.
For example, the simulated raising of the seat, by a user's hand, activates the letter b on the keyboard 156, the activated letter b, instructs the computer 156 to raise the seat 160.
Kinect Speech Recognition The Kinect software programmed in the computer 156 or microcontroller 159 matches a word or phrase to a system function. The speech recognition programming software listens for speech grammar, words that are associated with system functions. If the user says the word up, and the seat 160 and lid 162 are resting on the toilet bowel 192 in the down position, the associated system function is to have the computer move the seat and lid to the up position. Using the Kinect speech recognition software, the Kinect system 156 connects to the internet. Combined with the internet search engine Bing made by Microsoft, the system 156 will retrieve information from the internet. The internet can be accessed verbally using the microphone 155, shown in fig 20A.
The retrieved text information is converted to synthetic speech conveyed to the user through the speaker 164. Internet audio sound is conveyed through the speaker 164.
Kinect Assembly Holographic Internet Web Browsing The internet can be accessed using the touch sensitive holographic projector 195, illustrated in fig 20A. The holographic projector195 projects on to a surface or into space. The internet holographic display 195 is similar to a computer touch screen display presentation, displayed on a surface or into the air. The user touches the holographic images, touchable holograms 195, of the internet web pages, or displayed keyboard, to navigate the internet. Internet information is displayed holographically 195;
web pages, internet video internet TV, internet games, sound is broadcast through the speaker 164. User videos on a secure digital card can be placed in the secure digital card reader 77, and can be played and viewed holograpically 195, shown in fig 25. The system menu options can be accessed by touching the touch sensitive holographic display 195. Using the holographic touch screen in the air 195, or on a known sanitized surface, allows the user to access the system and internet without having to touch surfaces that may contain bacteria and viruses, Voice And Facial Recognition Seat And Lid Description An alternate embodiment uses a computer 98. The computer 98 is a net book portable computer 98, made by Samsung Company. The computer 98 is show by the abbreviation CPU 98, with CPU circled in Figs 7B and 9B. The computer 98 is programmed with voice recognition software 98. The speech recognition or voice recognition software is made by Sensory Company. The computer is programmed to rotate the servo motor shaft 89 and exact number of degrees, and to stop the rotation when the servo shaft 89 has reached the desired shaft. The computer 98 is programmed to turn off the servo motor 89 or servo motors 89, when the seat 47 and or lid 43 reach the up or down position. The rotational sensor 88 signals the computer 98, when the seat 47 or lid 43 reach the up or down resting position. In figs 7A
and 7B, a ultrasonic motion detector sensor 126 connects to the computer 98. The distance at which the ultrasonic sensor 98 activates the system is 2 to 3 meters, and is adjustable, The ultrasonic sensor 126, looks out of the front of the housing 196 through a transparent window or lens 126, a position whose view is unobstructed, whether the seat 47 or lid 43 are in a down position or in a up position (Fig 6B). A web cam, video camera, or video cam 61 connects to the computer 98. The web cam, looks out of the front of the housing 196, with an unobstructed view. A computer 98 reset button 67, on the right side on the housing 53 connects to the computer 98. A low battery indicator light 65, connects to the motor controller 99, and computer 98 and is located on the right side 53 of the housing. The motor controller 99 connects a rotating circuit 90 attached to a rotating shaft 79 of the lid 4. A wire 92 travels from the rotating circuit 90, through the inside of the shaft 79, through the inside of the lid 43, to 7 ultraviolet light emitting diodes 100, attached to the underside of the lid 43 The lid 43 connects to the rotating shafts 79, which travels through a washer 85, a lubricated o ring 81, into the housing, through two plastic ball bearing supports 83, having a rotation sensor 88 attached to the shaft 79. The lid 43 opposing shaft end 79 attached to the servo motor 89 is a similar fashion, as the seat shaft 79 connections. The system menu can be accessed either, verbally through the water resistant microphone 54 and water resistant speaker 55, or visually, using a liquid crystal display 114, with a menu control button. The menu control button also functions as a system on off button 116. A secure digital card reader 73 connects to the computer 98. The secure digital card reader 73 is located on the left side of the housing 51. It has a water resistant rubber flap 75 that covers the access port. Secure digital cards (not shown) with the user's music or sound effects can be placed into the reader 73. The assembly connects to broadband internet, by using a Wireless Fidelity card, Wi-Fl card 104 to wirelessly connect to the internet.
The assembly also uses a Ethernet connection 102 to connect to the internet by wire, as shown in Fig 9B. The Ethernet wire (not shown), plugs into the assemblies Ethernet outlet 102.
The motors 89, plastic ball bearings 83, rotation sensor 88, rotating circuit 90, ultra sonic sensor126, video cam 61, servo controller 99, microphone 54, secure digital card reader 73, Ethernet plug 102, computer or microcontroller system reset button 67, low battery indicator light 65, ACDC plug 71, are attached to the housing ,as shown in Figs 3A,7A and 17B.
Voice And Facial Recognition Seat And Lid Operation Figs 6A, 6B and 9A, shows the low power ultrasonic motion detector 126, which signals the computer 98, when a user is within 1.5 meters of the seat assembly 47, the system wakes up, and activates the system, coming out of low power and user detection mode. The system can also be activated, by a user's voice commands, if the system is in low power listening mode. The 1.5 meter activation distance can be adjusted by the user, through the menu options. The ultrasonic sensor 126 is located in the front of the housing 196, looking out through a transparent window 126.
The ultra-Sonic's 126 view is unobstructed weather the seat 47 and lid 43 are up or down. On initial contact the user hears an audible greeting of, nice day, or prompt of, what seat 47 position would you like. Etc. The user's reply to prompts, is analyzed by the computer's speech recognition software 98, voice recognition software 98. The system can respond to commands at distances up to 5 meters, or in high noise conditions. The computer 98, uses two speech engines, speech recognition, which the computer 98 hears and understands human speech, and speech synthesis, where the computer 98 talks, as text to speech capabilities.
Both speech engines used together, by the computer 98 (Fig 9B) to process a person's commands, e.g. seat 47 up, and synthesize speech in return. The computer's 98 software follows steps in understanding the users vocalized desires, Step 1, voice activation, high noise immunity, low power, step 2, speech recognition and transcription, moderate noise immunity, high accuracy, step 3, intent and meaning, accurately understands users desires, step 4, data search and query, the system finds the right information, step 5, the system accurately interprets text data, system speech response, sounds like a human, system function response directs servo motors movements and system functions.
The user can choose from an audible menu of seat 47 and lid 43 position options.
System menu functions can be accessed audibly using the microphone 54 and speaker 62, or visually using the liquid crystal display 114, as shown in Figs 6A, 6B
and 19. The user can move the seat 47 and or lid 43 by spoken words, of up or down, lid 43 up or lid 43 down, seat 47 up or seat 47 down, and stop. The user can chose from combinations of seat 47 positions, with seat 47 position down, resting on the toilet bowel 128, or seat 47 up, just past vertical resting on the lid 43.A combination with the lid 43 resting on the seat 47, or up resting against the toilet water tank 128, toilet hardware, or the assembly.
Other combinations are lid 43 up with seat 47 down, seat 47 up with lid 43 up, seat 47 down and lid 43 down. If the seat 47 and lid 43 are in a down position, the spoken word 'up' will move both seat 47 and lid 43 to the up position. Saying lid 47 up will move the lid 43 to the up position. Speaking the word down, will move the seat 47 and lid 43 to the down position. Saying seat down, will move the seat 47 to the down position, with lid 43 staying in the up position The user can say stop while the seat 47 is moving, which stops the seat.47 After the stop command is given, the seats 47 movement will stop and reverse direction The system can be trained to except different words for different system functions, such as; the word blue can be used for the word up, up to move the seat 47 to the raised position. The seat 47 and lid 43 positions can be moved manually, by hand in the usual manner.
In Figs 7A, 7B, the computer 98 signals the servo motor controller 99 to send a voltage to the servo motor 89 to rotate the motor shaft 89, an exact predetermined number of degrees, 95 degrees clockwise or counter clockwise. The servo motor shaft 89, is connected to the slip clutch 118, the slip clutch is connect to the rotating shaft 79 that's connected to the toilet seat 47, the rotation cause's the seat to pivot from horizontal, 90 degrees, to just past vertical, 05 degrees, or cycle in the other direction, from up to down. The lid 43 uses the same motor movement system, as the seat 47.
The slip clutches 118 exerts a minimal amount of force on a user, if they come in with contact the seat 47 or lid 43 while their moving. If the system activates while a user is on the seat 47 the computer 98 detects the seat 47 or lid 43 are in a stalled condition, by the increased motor voltage load, in Figs 7A, 7B, 9A and 9B. With movement stalled the computer 98 instructs the motor controller 99 to stop the seat 47 or lid's movement, and reverse their direction, then to stop their movement upon reaching their up or down resting place. The computer 98 is programmed to turn off the servo motor 89 or servo motors 89, when the seat 47 and or lid 43 reach the up or down position.
The rotational sensor 88 signals the computer 98, when the seat 47 or lid 43 reach the up or down resting position. When the computer 98 receives signals that the seat 47 or lid 43 have reached their up or down position, the computer 98 signals the motor controller 99 to stop the motor 89 or motors 89, which stops the seat 47 or lids 43 movements. Both the seat 47 and lid 43 can move in tandem, or move independently of each other. When the seat 47 and lid 43 are in different positions, seat 47 down and lid 43 up, the system carries out one function or movement at a time, to avoid the seat 47 and lid 43 acting against each other, such as the seat 47 moving up and the lid 43 moving down. The speed that the seat 47 or lid 43 move up or down, cycles up or down, can be adjusted by the user, in the menu options, e.g. slow, medium, fast.
Music And Sound Effects Seat And Lid Operation When a user is within 1.5 meters of the seat 47, music or sound effects can be played, also the system may converse with the user, by using microphone 54 and speaker 55 as illustrated in Fig 9A. The volume of the music, greeting or sound effects, can be lowered by accessing the options menu, going to sound and music, going to volume, saying raise or lower. Conversing with the system may relax a person, and aid in elimination, which may help maladies, such as constipation. Conversing with the system may be fun and informative. Conversing with the system may, result in a positive bond developing for the user toward the system.
Music or sound effects may cover up the sound of elimination. The user may play music or sound effects through the speaker. The user may choose music or sound effects from the system menu, or access their own imported music. A user may import music, by placing a secure digital card with the user's music stored on the card, in the assembly's secure digital card reader 75, abbreviation SD card reader. The SD
card reader 75 is connected to the computer 98, as shown in Fig 7B. A user can access the card reader on the left side of the housing 51. The SD card reader 73 has a water resistant flap 75, covering the access port. The reader can be accessed 73 by lifting the rubber flap 75, and placing the SD card in the reader. Once the SD card is in the reader 75 the user can replace, the waterproof rubber protective covering75, over the reader's access port opening 73 (Fig 9B).
Accessing System Menu Seat And Lid Operation The system menu can be accessed verbally through the microphone 54 and speaker 62, shown in Fig 6A. The system can also be accessed through a liquid crystal display114, and menu control button 116 as shown in Fig 6A. Fig 9A shows the menu control button 116, which can be pressed left, right, up, down and straight down.
Pressing the button 116, in Fig 9A, up or down scrolls through the menu options, possible options include music, sound effects, volume control, and seat 47 and lid 43 rotation speeds, greetings, UV LEDs 100 time on duration, ultrasonic 126 user distance to activate the system, facial recognition on or off. Pressing the button 116 to the right direction or would start the menu at a category and travel to smaller parts of the category, such as for sound effects, pressing the button 116 right would get to specific actions, such as type of music to play, such as dance music, ocean wave sound etc.
Pressing the button down 116, in the center would choose the displayed option.

Pressing the button 116 left would collapse the menu options, returning to the main category and returning to the main menu. Pressing the center of the button for a 4 second time interval, turns the system on and off. The audible menu follows the same menu path as the LCD display 114 menu path, with the system audibly announcing the menu options.
Power Supply Seat And Lid Operation Electrical power to the assembly, as shown in Figs 9A, 9B and 15A, is supplied by either the lithium ion battery 172, or by the ACDC adapter 72, which plugs into an electrical power supply outlet (not shown). When the ACDC power adapter 72 supplies power to the system, power from the battery 172 automatically shuts off. The low battery indicator light 65 is connected to the computer 98 and motor controller, which illuminates when the battery 172 needs to be recharged. The battery 172 can be charged in the assembly by choosing charge battery, from the system menu, and by plugging the ACDC adapter 72 into the assembly plug 71. The battery can be charged outside of the assembly, by removing the battery 172, by pressing the battery release button 176, and plugging the ACDC adapter plug 72 into the battery 172, shown in Fig 15A.
Ultraviolet Light Emitting Diodes Radiating Seat And Lid Operation The ultra-sonic 126 sensor and computer 98 detect when a user, has vacated the seat 47 (Fig 9A), the assembly then moves the seat 47 to the preferred user vacant seat position. The ultra-sonic 126 sends signal to the computer 98, when they detects that a user has vacated the area for 30 seconds, a user adjustable time period, the computer 98 puts the system in low power wait mode, leaving the low power ultrasonic 126 active.
The system also turns on 7ultraviolet light emitting diodes 100, UV LEDs, attached to the underside of the lid 43, in Fig 6B, for 8 minutes, the time is adjustable by the user.
The UV LEDs 100 turns on only if the lid 43 is resting on the seat 47. The UV
LEDs 100 radiates the seat 47, lid 43, toilet bowel 128 and toilet bowel water. UV
radiating reduces bacteria and viruses, on their surfaces. Radiating reduces bacteria in the toilet bowel water, which may splash on a user during elimination. Reducing bacteria and viruses, reduces the spread and transmission of e coli bacteria and hepatitis viruses. If a user is detected, within 1 meter of the toilet seats 47 area, while the UV
LED lights 100 are on, the system turns off the LED lights 100.
Internet Connection Seat And Lid Operation The assembly connects to broadband internet, by using the Wi-Fi card 104 to wirelessly connect, or the Ethernet cable connection, to connect by wire, as shown in Fig 9B. The Ethernet cable (not shown), plugs into the assemblies Ethernet plug 102, fig 9B. The internet connection allows a person to verbally request information, through the microphone 54, and hear the requested information, through the speaker 62, Fig 6A.
The internet is accessed verbally, through the system menu, and internet information is conveyed to the user audibly, through the speaker 55. The user may access information from the internet', such as, traffic conditions, news, email, etc.
ADDITIONAL ALTERNATE EMBODIMENTS
Interchangeable Toilet Seat Description Figs 13A, 13B, 16A and 16B shows an Inter changeable seat 142 that attaches to the bases two shafts. A right shaft 138 that use a spring 140 inside the shaft that allows the shaft to be compressed. The right seat support 142 attaches to the right side assembly shaft.138 A left shaft 150 that attaches to seat's left support 142.
The left shaft 150 has a indent 150 that the seat's left supports 142 locking mechanism secures to, securing the seat 142 to the assembly. The left seat support 142 has a lock to secure the coupling of the seat support 142 and shaft 138. A quick release unlock lock button146 is part of the seat support locking mechanisms, Figs 17A and 17B.
Interchangeable Toilet Seat Operation An alternate embodiment Figs 13A, 13B, 16A and 16B shows a Interchangeable toilet seat 142 that attaches a base148, the base attaches 148 to a toilet (not shown).
The seat connects to a base by two shafts 138, 150, by compressing the right shaft 138, then positioning the other seat support 142 in the opposing left shaft 150.
The seat support 142 has a lock to secure the coupling of the support 142 and shaft with a shaft indent 138. A quick release button146 is pressed to release the seat 142 from the shaft 150, Figs 17A and 17B. The interchangeable toilet seat 142 can be changed, for different seasons, or changed for variety, etc.
One Pivot Connection Point Seat And Lid Description An alternate embodiment as show in Figs 12A, 12B, 14A, and 14B, utilizes a one support toilet seat 132 and one support lid 134, which connects to shafts 135 that are connected to the base 48, which is attached to the toilet 128. The one support seat 132 attaches to the right shaft 135 using a cap 136 that screws on to the shaft 135. The seat is secured to the right shaft 135 which is connected to the base 49, which is attached to the toilet 128. The lid 134 attaches to the left shaft 135 in a similar manner as the one support seat 132. The seat 132 and 1id134 connect to the base at a one pivot connection point, to the base 49. Fig 14A shows a permanent anti-bacterial coating 154 on the seat.
One Pivot Connection Point Toilet Seat Operation An alternate embodiment as show in Figs 12A, 12B, 14A, and 14B, utilizes a one support toilet seat 132 and one support lid 134 that connects to a shaft 135 that connects to the base 49, which is attached to the toilet 128. The one support seat 132 attaches to the base 49 using caps 136. The caps 136 screw on the threads of the shafts 135, securing the seat 132 and lid 134 to the base 49.The seat 132 and 11d134 connect to the base 49 at a one pivot connection point, to the base 49. The one support seat 132 and lid 134 are attractive to users interested in different seat 132 and lid 134, styles, connections and configurations, to the seat and lids base 49. Fig 14A
shows a permanent anti-bacterial coating 154 on the seat, which reduces bacteria on the seats surface.
The toilet seat assembly 132 may be made out of water resistant high strength plastic, wood, metal, rubber, and the like..
Attaching The Assembly To The Toilet Attaching the assembly to the toilet 128 is an easy, do it yourself installation, for the user. Figs 18A and18B shows a plastic plate 152, that allows for two bolts 76 and nuts 74 that can slide width wise, to adjust for differing toilet bowel hole distances, when attached to the toilet. Figs 4A and 4B shows a plate 124, which is also available, that allows the two bolts to slide longitudinally. The assembly slides on to the anchor plate.
Figs 4A, 4B, 18A and 18B show how the plate is secured to the assembly by sliding a latch down 130, thus securing the assembly to the toilet 128.
Safety Factors Safety factors include the circuit breaker and the circuit breaker test button, incorporated into the ACDC adapter electrical power plug 72, Figs 9A, 15A. The slip clutches 118 only exerts a minimum amount of force applied to a user, who contacts the seat 40 or lid 42 while they're moving, shown in Figs, 3A and 7A. The computer detects when a motor 86 voltage load increases, caused by the stalling of the seats 40 or lids 42 movement, Figs 7B and 8A. When contacting a user the computer 98 signals the servo controller 96 to stops the seat 40 or lids 42 movement, and reverses their direction till they stop in the up or down resting position, shown in Fig. 9B.
If the system activates the seat 40 or lids 42 movement, while a user is sitting on the seat 40, the system stops the seat 40 or lids 42 movement. If the word, stop is vocalized while the seat 40 or lid 42 are moving, the seat 40 or lid 42 will stop, reverse direction, and move to their up or down resting position.

ADVANTAGES:
From the description above, a number of advantages of some of my, voice recognition, facial recognition and gesture recognition, toilet seat and lid, raising and lower assembly become evident.
(a) The voice recognition toilet seat reduces the spread and transmission of bacteria and viruses form the toilet seat to a user's hands, to the user and to other people.
(b) The voice recognition system is more versatile, user friendly, simpler and easier to use, then other systems, it only needs speech to operate. A user can, be identified by their voice. Once the system recognizes a user, by recognizing their voice, the system sets seat and lid preferred positions and system functions, such as music, an internet connection:
(c) The device may use either facial recognition software, and or voice recognition software, to identify a user, and set the users system, seat position, and music preferences.
(d) Using facial recognition software, in the assembly, lets the system to identify a user, by viewing their facial image. Once the user is identified, the seat is then moved to a user's preferred seat up or down position. The user only needs to be within sight of the assembly to have the assembly move the seat to the users preferred position.
(e) The toilet seat assembly reduces the need to bend over to position the seat, which may be helpful for disabled or elderly users.
(f) Body and gesture recognition software, enables a user to position the toilet seat and lid using hand gestures and body motions.
(g) Light emitting diodes attached to the underside of the lid; illuminate the top of the toilet seat, the underside of the lid, the toilet bowel, the toilet water, lessening bacteria on the surfaces, and in the water. Thereby, lowering the amount of bacteria that the user is contaminated with, when contacting these surfaces. The UV LED's lessen bacteria in the toilet water that may come in contact with a user, by the splashing of toilet water during urination and bowel elimination.
(h) It may be fun and enjoyable to converse with the system, through the microphone and speaker interface, listening to greetings, prompts, and talking with the system, this conversing, may lead to a positive bonding between the user and the apparatus. Vocalizations, within 4 to 5 meters, can operate the system.
(i) Advantages over traditional toilet seats are the assembly plays music, that may cover up the sound of urinary and bowel elimination. The user can also, input their music, on the device through a secure digital card reader. The device plays music and sound effects, which may relax a user, and aid in the process of elimination that may be helpful concerning constipation and other ailments.
(j) The surface of the toilet seat is permanently infused with an anti-bacterial technology substance, which reduces germs on the seats surface.
(k) Attaching the seat assembly, to the toilet bowel is an easy do it yourself installation, for a consumer.
(I) The assembly's operation, lowers interpersonal relationship tension, between different sexes. Having the assembly automatically positioning the seat up or down, removes the need for different sexes to communicate their seat position desire to each other, e.g. please put down the seat after using.
(m) The assembly reduces the need to inform alternate users, to leave the toilet seat in a desired position, such as, put the seat down when you're finished.
Users don't have to remember, to the leave the seat in a certain position, for other users. Users don't have to touch the toilet seat, to put it in a preferred position, for an alternate user, thus reducing friction between users, and possibly creating a more harmonious environment.

(n) The system may have an internet connection, to allow for the timely import of information and data, which the user can verbally access and aurally receive.
(o) The assembly incorporates safety factors for users. A slip clutch limits the amount of force a user, may be exposed to if they contact the seat or lid, while they are moving. If the word stop is spoken, the device will stop the seat's movement, and it will reverse direction. If the seat or lid meets resistance, while moving the micro controller or computer stops the seat or lids movement, and reverses their direction. If the seat or lid activate, while a user is sitting on it, the system will stop the seat and lids movement.
The device has a circuit breaker, and a circuit breaker test button, incorporated within the alternating current to direct current adapter.
(p) The seat assembly incorporates interchangeable toilet seats. The assembly may incorporate an interchangeable lid (not shown). Toilet seat styles, and colors, can be easily changed. Different styles of seat and lid such as an interchangeable warm seat in the winter, cooler seat in the warmer summer season. Different seats can be used for different occasions, such as for St. Patrick day, a green colored seat, or for the 4th of July a seat with fireworks designs on it, or seats that maybe changed for variety.
(q) The system menu can be accessed verbally using the microphone and speaker, visually using the liquid crystal display, or using body gestures.
(r) The holographic projector displays touch screen holographic projections.
The user can interact with the computer by touching the holographic display screen, on a surface or in the air, in the same manner as using a touch screen table computer.
Touching the holographic screen display in space or on a hygienic surface, reduces the transmission of bacteria and viruses. The holographic display enable a user to surf the Internet, view internet web pages, view internet content and listen to internet videos.
The user can chose system functions using the holographic display. The seat assembly's holographic functions can be fun and sanitary.
CONCLUSION

Although the description above contains much specificity, these should not be construed as limiting the scope of the embodiments but as merely providing illustrations of some of the presently preferred embodiments. For example the seat can have different shapes, such as, a toilet seat with a notch in the front, the seat being u shaped.
UV LEDs may be attached to the housing, radiating, the toilet seat, toilet bowel, and toilet water. The assembly may incorporate user, motion sensing devices, body and hand gesture recognition software, which can be made by the Sony Company, Nintendo Company, Samsung Company, etc. Rotary potentiometers and optical encoders maybe used, to sense the rotation and position of the seat or lid. A bidet may be incorporated in the assembly .The assembly can be made of different materials, such as wood, plastic, metal, etc. The seat maybe electrically warmed. The assembly may use various rechargeable batteries, such as, NiCad rechargeable batteries or Nickel Metal Hydride, NiMH, rechargeable batteries. The microcontroller or computer, can be made by the Adrino Company, and programmed with Adrino robotics software. The computer may be made by many possible computer companies such as, Sony Company or Hewlet Packard Company. The computer may be a tablet computer. The computer may use microprocessors made by Advanced Micro Devices Company, or Intel Company. The computer could be programmed with various speech recognition software made by the Microsoft Company or Nuance Company. Apple Company's, Sin i speech recognition software may be incorporated into the assembly. A brain wave sensing device, may be used to allow a user to control the up and down movements of the toilet seat.
A infrared sensor can be used to detect users. Facial recognition software used to program the system, could be made by the Identx Company, PittPatt Company, and Sensible Vision Company. The visual system menu options screen might use a touch sensitive screen, to choose menu options. Many types of flash memory card or storage cards can be used to import music, greetings, and sound effects, such as, a Memory Stick, a CompactFlash, or a MultiMediaCard. A Bluetooth transmitter and receiver may be incorporated in the assembly.
Thus the scope of the embodiments should be determined by the appended claims and their legal equivalents rather than by the examples given.

Thus the reader will see that at least one embodiment of the toilet seat assembly provides a more reliable, healthier and economical device that can be used by persons of almost any age. It will be apparent that various changes and modifications can be made without departing from the scope of the various embodiments as defined in the claims.
Although the embodiments have been described in connection with various embodiments, it should be understood that various modifications, additions and alteration may be made to the embodiment by one skilled in the art without departing from the spirit and scope of the embodiments as defined in the appended claims.
Provisional Patent Application of sigmund lindsay clements for, 'Voice Recognition Responsive And Motion Sensing Touch Free Toilet Seat Raising And Lowering Assembly'.
Address 4822 rue de la Peltrie , montreal , Quebec, H3W 1K5 , Canada, 514-733-Additional programming descriptions for the Kinect sensing device, connected to the computed 56, or microcontroller 159, as shown in fig 22A, 24, 25.
Speech Recognition (Microsoft.Speech) The Microsoft.Speech.Recognition namespace provides functionality with which you can acquire and monitor speech input, create speech recognition grammars that produce both literal and semantic recognition results, capture information from events generated by the speech recognition and, and configure and manage speech recognition engines.
Speech Input With the input functionality of speech recognition, your application can monitor the state, level, and format of the input signal, and receive notification about problems that might interfere with successful recognition. See Audio Input for Recognition (Microsoft.Speech).
Grammars You can create grammars programmatically using constructors and methods on the GrammarBuilderand Choices classes. Your application can dynamically modify programmatically created grammars while it is running. The structure of grammars authored using these classes is independent of theSpeech Recognition Grammar Specification (SRGS) 1Ø See Create Grammars Using GrammarBuilder (Microsoft.Speech).
To create dynamic grammars programmatically that conform to the SRGS
specification, see Create Grammars Using SrgsGrammar (Microsoft.Speech). The classes of theMicrosoft.Speech.Recognition.SrgsGrammar namespace map closely to the elements and attributes of the SRGS specification.
You can also create grammars as static files, using SRGS-compliant XML markup, that your application can load at runtime. See Create Grammars Using SRGS XML
(Microsoft.Speech).
Use the constructors on the Grammar class to compile grammars created with any of the above methods into Grammar objects that the speech recognition can load and use to perform recognition.
Semantics Recognition engines use the semantic information in grammars to interpret recognition results. To add semantic information to programmatically-created grammars, see Add Semantics to a GrammarBuilder Grammar (Microsoft.Speech). You can add semantic information to XML-format grammars using ECMAScript (JavaScript, JScript) in the tag elements. See Semantic Interpretation Markup (Microsoft.Speech). For information about semantic results returned by speech recognition engines, see Create and Access Semantic Content (Microsoft.Speech).
Debugging Tools The Microsoft Speech Platform SDK 11 includes a comprehensive suite of command-line tools with which you can test, debug, and optimize your voice applications without first deploying them to a live service. See Microsoft Grammar Development Tools for more information.
In addition, Microsoft.Speech has special emulated recognition modes that allow you to provide text instead of audio to the speech recognition engine. You can use the simulated recognition results for debugging and optimizing speech recognition grammars. See Emulate Spoken Commands (Microsoft.Speech).
Events Your application can register for events that the speech recognition engine generates when completing important stages during speech processing such as loading grammars or recognizing speech. See Use Speech Recognition Events (Microsoft.Speech).
Recognition Engines Using the members of the SpeechRecognitionEngine class, you can initialize a speech recognition engine instance, select an installed Runtime Language to use for recognition, load and unload grammars, subscribe to speech recognition events, configure the audio input, start and stop recognition, and modify properties of the speech recognition engine that affect recognition. SeeInitialize and Manage a Speech Recognition Engine (Microsoft.Speech).
In addition, the speech engines for Microsoft.Speech will recognize the dual-tone multi-frequency (DTMF) tones that are important for developing applications that interact with users by telephone. SeeDTMF Recognition.
How Speech Recognition Works (Microsoft.Speech) A speech recognition engine (or speech recognizer) takes an audio stream as input and turns it into a text transcription. The speech recognition process can be thought of as having a front end and a back end.

Convert Audio Input The front end processes the audio stream, isolating segments of sound that are probably speech and converting them into a series of numeric values that characterize the vocal sounds in the signal.
Match Input to Speech Models The back end is a specialized search engine that takes the output produced by the front end and searches across three databases: an acoustic model, a lexicon, and a language model.
= The acoustic model represents the acoustic sounds of a language, and can be trained to recognize the characteristics of a particular user's speech patterns and acoustic environments.
= The lexicon lists a large number of the words in the language, and provides information on how to pronounce each word.
= The language model represents the ways in which the words of a language are combined.
For any given segment of sound, there are many things the speaker could potentially be saying. The quality of a recognizer is determined by how good it is at refining its search, eliminating the poor matches, and selecting the more likely matches. This depends in large part on the quality of its language and acoustic models and the effectiveness of its algorithms, both for processing sound and for searching across the models.
Grammars While the built-in language model of a recognizer is intended to represent a comprehensive language domain (such as everyday spoken English), a speech application will often need to process only certain utterances that have particular semantic meaning to that application. Rather than using the general purpose language model, an application should use a grammar that constrains the recognizer to listen only for speech that is meaningful to the application. This provides the following benefits:
= Increases the accuracy of recognition = Guarantees that all recognition results are meaningful to the application = Enables the recognition engine to specify the semantic values inherent in the recognized text The Microsoft Speech Platform SDK 11 provides processes for authoring grammars programmatically, and also supports grammars authored using industry-standard markup language.
Get Started with Speech Recognition (Microsoft.Speech) A speech recognition application will typically perform the following basic operations:
= Start the speech recognizer.
= Create a recognition grammar.
= Load the grammar into the speech recognizer.
= Register for speech recognition event notification.
= Create a handler for the speech recognition event.
The following provides information about how to program each of these operations.
Start the Speech Recognizer To start the speech recognizer, create a new SpeechRecognitionEngine instance.
The following example uses a constructor whose parameter specifies the culture of the recognizer that the speech recognition engine should use for recognition, in this case US English (en-US). If the constructor does not specify a culture, the SpeechRecognitionEngine instance will use the default recognizer on the system.
The example does not show the declaration of the sre object.
C#
// Create a new SpeechRecognitionEngine instance.
sre = new SpeechRecognitionEngine(new System.Globalization.CultureInfo("en-US"));
Note A recognizer is an installed Runtime Language for speech recognition. A
Runtime Language includes the language model, acoustic model, and other data necessary to provision a speech engine to perform speech recognition in a particular language.
See InstalledRecodnizers() for more information.
Create a Speech Recognition Grammar One way to create a speech recognition grammar is to use the constructors and methods on theGrammarBuilder and Choices classes. The following example creates a simple grammar that recognizes the words "red", "green", or "blue". The words are added using a Choices object. For a match between user speech and the grammar to occur, the user must speak exactly one of the elements added by the Choices instance.
Each of the words is added by a call to the Add(StringMmethod.
After the Choices instance is created and set with the option strings, the example creates aGrammarBuilder instance. Using the Append(Choices) method, the example appends the colors object to the GrammarBuilder instance. In the last line, the example creates a Grammar instance and initializes it with the GrammarBuilder instance.
C#

Choices colors = new Choices();
colors.Add(new string[] {"red"});
colors.Add(new string[] {"green"});
colors.Add(new string[] {"blue"});
GrammarBuilder gb = new GrammarBuilder();
gb.Append(colors);
// Create the Grammar instance.
Grammar g = new Grammar(gb);
Alternatively, the first four lines of code can be collapsed to a single line by using the Choicesconstructor to initialize the instance with an array of string options, as follows:
Choices colors = new Choices(new string[] {"red", "green", "blue"});
For more information about creating grammars, see Create Grammars (Microsoft.Speech).
Load the Grammar into a Speech Recognizer After the grammar is created, it must be loaded by the SpeechRecognitionEngine instance. The following example loads the grammar by calling the LoadGrammar(Grammar) method, passing the grammar created in the previous operation.
c#
sre.LoadGrammar(g);
Register for Speech Recognition Event Notification The SpeechRecognitionEngine object raises a number of events during its operation, including theSpeechRecognized event. For more information, see Use Speech Recognition Events (Microsoft.Speech). The SpeechRecognitionEngine instance raises the SpeechRecognized event when it matches a user utterance with a grammar. An application registers for notification of this event by appending an EventHandler instance as shown in the following example. The argument to theEventHandler constructor, sre SpeechRecognized, is the name of the developer-written event handler.
c#
sre.SpeechRecognized += new EventHandler<SpeechRecognizedEventArgs>(sre_SpeechRecognized);

Create a Speech Recognition Event Handler When you register a handler for a particular event, the Intellisense feature in Microsoft Visual Studio creates a skeleton event handler if you press the TAB key. This process ensures that parameters of the correct type are used. The handler for the SpeechRecognized event shown in the following example displays the text of the recognized word using the Result property on the SpeechRecognizedEventArgsparameter, e.
C#
void sre_SpeechRecognized(object sender, SpeechRecognizedEventArgs e) {
MessageBox.Show(e.Result.Text);
}
Speech Recognition Example The following examples are components of a Windows System.Windows.Forms application that features speech recognition.
Although the application displays a form when it starts, nothing of interest happens with the form.
The first example contains the code for the handler for the form's Load event, which is raised when the form is loaded.
Almost everything of interest in this application occurs in the Form l_Load method.
The method builds a grammar incrementally using a Choices instance to add the strings "red", "green", and "blue". It then creates a GrammarBuilder instance using the Choices object. The method then initializes a Grammarinstance with the GrammarBuilder object created earlier. The grammar, which is capable of recognizing the words "red", "green", or "blue", is then loaded into the speech recognizer. Finally, theForml_Load method registers an event handler for the SpeechRecognized() event.
The sre SpeechRecognized method, which executes when the SpeechRecognitionEngine instance raises the SpeechRecognized() event, displays whichever of the three colors the user spoke. The following illustration shows the interaction between the user's speech and theSpeechRecognitionEngine with its grammar. When the utterance matches an element in the grammar, as shown in fig the speech recognizer makes a recognition, and produces a recognition result.
C#
using Microsoft .Speech.Recognition;
using System;
using System. Collections . Generic ;

using System.ComponentModel;
using System.Data;
using System.Drawing;
using System.Ling;
using System.Text;
using System.Windows.Forms;
namespace WindowsFormsApplicationl public partial class Forml : Form public Forml() InitializeComponent();
private void Forml_Load(object sender, EventArgs e) // Create a new SpeechRecognitionEngine instance.
sre = new SpeechRecognitionEngine();
// Create a simple grammar that recognizes "red", "green", or "blue".
Choices colors = new Choices();
colors.Add(new string[] Fired", "green", "blue"));
// Create a GrammarBuilder object and append the Choices object.
GrammarBuilder gb = new GrammarBuilder();
gb.Append(colors);
// Create the Grammar instance and load it into the speech recognition engine.
Grammar g = new Grammar(gb);
sre.LoadGrammar(g);
// Register a handler for the SpeechRecognized event.
sre.SpeechRecognized +=
new EventHandler<SpeechRecognizedEventArgs>(sre_SpeechRecognized);

// Create a simple handler for the SpeechRecognized event.
void sre_SpeechRecognized(object sender, SpeechRecognizedEventArgs e) {
MessageBox.Show(e.Result.Text);
SpeechRecognitionEngine sre;

The following example is autogenerated code for a Windows Forms application.
c#
using System;
using System.Collections.Generic;
using System. Ling;
using System .Windows . Forms;
namespace WindowsFormsApplication1 static class Program // The main entry point for the application.
[STAThread]
static void Main() Application. EnableVisualStyles ( ) ;
Application .SetCompatibleTextRenderingDefault(false);
Application.Run(new Form1());
Benefits of Using the Microsoft Speech Platform SDK 11 Speech is an effective and natural way for people to interact with applications, complementing or even replacing the use of mice, keyboards, controllers, and gestures.
A hands-free, yet accurate way to communicate with applications, speech lets people be productive and stay informed in a variety of situations where other interfaces will not.
Microsoft Speech Platform SDK 11 The Microsoft Speech Platform SDK 11. includes both managed-code and native-code application programming interfaces (APIs).
The Microsoft.Speech managed-code namespaces provide you with easy access to the advanced speech recognition and speech synthesis technologies supported by the Microsoft Speech Platform Runtime 11. The functionality that the managed-code APIs expose is a subset of the native-code APIs and will satisfy the majority of speech application scenarios.
The native-code APIs implement all the low-level details needed to control and manage the real-time operations of various speech engines.

You do not need to create your own engines for speech recognition and text-to-speech because you can download and install the speech Runtime and any of the 26 Runtime Languages for speech development. See What's New in the Microsoft Speech Platform SDK 11.
Speech Recognition Speech recognition allows users to interact with and control your applications by speaking. Using the APIs in the Speech Platform SDK 11, you can acquire and monitor speech input, create speech recognition grammars that produce both literal and semantic recognition results, capture information from events generated by the speech recognition, and configure and manage speech recognition engines. See Speech Recognition (Microsoft.Speech).
Speech Synthesis (text-to-speech or TTS) The speech synthesis APIs in the Speech Platform SDK 11 will have your application talking with just a few lines of code. You get control over many aspects of voice expression, including volume, speaking rate, and pitch. See Speech Synthesis (Microsoft.Speech).
Standards Compatible For both speech recognition and speech synthesis, you get your choice of authoring formats. You can author content programmatically, using the members of the Microsoft.Speech namespaces. Or you can create industry-standard, XML-format speech documents for consumption by the Speech Platform Runtime 11. You can even alternate authoring formats based on your preferences and combine programmatic content with XML-format content.
Grammar Development Tools The Speech Platform SDK 11 provides tools that you can use to debug and tune grammars that you create for use with the Speech Platform Runtime 11. The functionality of the Microsoft Grammar Development Tools includes grammar validation, phrase generation, confusability detection, batch-mode simulated speech recognition, analysis of simulated recognition results including graphs, and grammar compiling. The Grammar Development Tools will help you to identify problem areas that are hard to detect through standard speech application testing, and to identify problems earlier in the application development life cycle. See Microsoft Grammar Development Tools.

Cost Efficient The Speech Platform SDK 11 can significantly reduce the amount of work necessary to implement speech in applications for Windows Server or other platforms that can leverage Microsoft's redistributable speech engines. By providing access to the Speech Platform Runtime 11 and engines for speech recognition and speech synthesis, the Speech Platform SDK 11 gives you the tools to create sophisticated speech applications while shortening development cycles and reducing development costs.
Microsoft Speech Programming Guide This programming guide provides an introduction to developing speech applications using the Microsoft.Speech namespaces in the Microsoft Speech Platform SDK 11.
The concepts and examples presented in this documentation will show you how to incorporate speech recognition and speech synthesis (text-to-speech or TTS) into applications running on Windows Server or other platforms that can leverage Microsoft's redistributable speech engines. We recommend that you read the sections titled Speech Synthesis and Speech Recognition before beginning a speech application.
In the section titled Speech Recognition (Microsoft.Speech), you will find discussion and examples that will help you accomplish the following:
= Create grammars = Initialize and manage a speech recognition engine = Interpret the results returned by a speech recognition engine = Work with speech recognition events The section titled Speech Synthesis (Microsoft.Speech) includes concepts and examples that will help you accomplish the following:
= Author content for TTS
= Generate speech in your application = Select and modify a voice = Work with speech synthesis events In addition, you will find reference sections to industry-standard XML markup languages including speech recognition, speech synthesis, semantics, and pronunciation lexicons, as supported by the Speech Platform SDK 11.
Speech C# How To Overview This How To contains user tasks from the Speech C# sample. The sample captures streaming audio from a Kinect using a speech recognition engine (sre). After you provide the grammar for the engine, the sample writes out hypothesized and recognized text to a command window. To see all the detail in the Speech C#
sample, use the Kinect for Windows SDK Sample Browser.
Code It Add a reference to the speech recognition assembly 1. Install the Microsoft Speech Platform SDK which is part of the Software Requirements for performing speech recognition using the Kinect for Windows SDK.
2. Open your C# project in Visual Studio 3. In Solution Explorer, expand the References node 4. Right click on References and select Add Reference 5. Browse to the path where the dll is located (CAProgram Files\Microsoft SDKs\Speech\v11.0\Assembly\Microsoft.Speech.d11) and click on Microsoft.Speech.d11 In addition, add the following using statement to the top of your .cs file.
using Microsoft.Speech.Recognition;
Initialize the audio source 1. Get the audio source 2. Disable echo cancellation 3. Disable automatic gain control(AGC must be disabled for speech recognition scenarios) KinectSensor sensor - (from sensorToCheck in KinectSensor.KinectSensors where sensorToCheck. Status == KinectStatus.Connected select sensorToCheck).FirstOrDefault();
using (var source = sensor.AudioSource) source. EchoCancellationMode = EchoCancellationMode.None;
//No AEC for this sample source .AutomaticGainControlEnabled = false;
= = =
Configured in this way, the microphone array functions as a single-directional microphone that is pointed within a few degrees of the audio source.

Initialize speech recognition Get a speech recognizer from the speech recognition engine.
RecognizerInfo ri = GetKinectRecognizer();
private static RecognizerInfo GetKinectRecognizer() Func<RecognizerInfo, bool> matchingFunc = r =>
string value;
r.AdditionalInfo.TryGetValue("Kinect", out value);
return "True".Equals(value, StringComparison.InvariantCultureIgnoreCase) && "en-US".Equals(r.Culture.Name, StringComparison.InvariantCultureIgnoreCase);
1;
return SpeechRecognitionEngine.InstalledRecognizers().Where(matchingFun c).FirstOrDefault();
SpeechRecognitionEngine.InstalledRecognizers is a static method that returns a list of speech recognition engines on the system. Speech uses a Language-Integrated Query (LINQ) to obtain the ID of the first recognizer in the list and returns the results as a RecognizerInfo object. Speech then uses RecognizerInfoid to create a SpeechRecognitionEngine object.
Create a speech recognition engine Speech uses command recognition to recognize three voice commands: red, green, and blue. You specify these commands by creating and loading a grammar that contains the words to be recognized.
1. Create the speech recognition engine (sre) 2. Create word choices 3. Build the grammar and load it in the sre 4. Register for recognize, hypothesize, and reject speech events using (var sre = new SpeechRecognitionEngine(ri.Id)) var colors = new Choices();
colors.Add("red");
colors.Add("green");
colors.Add("blue");
var gb = new GrammarBuilder();
//Specify the culture to match the recognizer in case we are running in a different culture.

gb.Culture = ri.Culture;
gb.Append(colors);
// Create the actual Grammar instance, and then load it into the speech recognizer.
var g = new Grammar(gb);
sre.LoadGrammar(g);
sre.SpeechRecognized += SreSpeechRecognized;
sre. SpeechHypothesized += SreSpeechHypothesized;
sre. SpeechRecognitionRejected +=
SreSpeechRecognitionRejected;

The Choices object represents the list of words to be recognized. To add words to the list, call Choices.Add. After completing the list, create a new GrammarBuilder object which provides a simple way to construct a grammar and specify the culture to match that of the recognizer. Then pass the Choices object to GrammarBuilder.Append to define the grammar elements. Finally, load the grammar into the speech engine by calling SpeechRecognitionEngine.LoadGrammar.
Each time you speak a word, the speech recognition compares your speech with the templates for the words in the grammar to determine if it is one of the recognized commands. However, speech recognition is an inherently uncertain process, so each attempt at recognition is accompanied by a confidence value.
Listen to user speech The speech recognition engine automatically attempts to recognize the words in the grammar and raises events as appropriate.
This using section does the following:
= Start streaming audio data from the Kinect sensor = Stream the data into the speech reconition engine = User controls what is drawn on the console by saying "red" or "green"
= The sre starts processing the speech using RecognizeAsynco = The sre stops processes the speech using RecognizeAsyncStop0 using (Stream s = source.Start()) sre.SetInputToAudioStream(s, new SpeechAudioFormatInfo(EncodingFormat.Pcm, 16000, 16, 1, 32000, 2, null) );

Console.WriteLine("Recognizing. Say: 'red', 'green' or 'blue'. Press ENTER to stop");
sre.RecognizeAsync(RecognizeMode.Multiple);
Console.ReadLine();
Console.WriteLine("Stopping recognizer sre.RecognizeAsyncStop();

Speech starts capturing audio from the Kinect sensor's microphone array by calling KinectAudioSource.StartCapture. This example also:
= Calls SpeechRecognitionEngine.SetInputToAudioStream to specify the audio source and its characteristics.
= Calls SpeechRecognitionEngine.RecognizeAsync and specifies asynchronous recognition. The engine runs on a background thread until the user stops the process by pressing a key.
= Calls SpeechRecognitionEngine.RecognizeAsyncStop to stop the recognition process and terminate the engine.
Respond to user speech In this example, the application chooses 0.7 as a confidence bar to accept a word as recognized. This value depends on each application and should be tested and evaluated on a case by case basis.
private void SreSpeechRecognitionRejected(object sender, SpeechRecognitionRejectedEventArgs e) this.RejectSpeech(e.Result);
private void SreSpeechHypothesized(object sender, SpeechHypothesizedEventArgs e) this.ReportSpeechStatus("Hypothesized: " + e.Result.Text + "
" + e.Result.Confidence);
private void SreSpeechRecognized(object sender, SpeechRecognizedEventArgs e) SolidColorBrush brush;
if (e.Result.Confidence < 0.7) this.RejectSpeech(e.Result);
return;
switch (e.Result.Text.ToUpperInvariant()) {
case "RED":
brush = this.redBrush;
break;
case "GREEN":
brush - this.greenBrush;
break;
case "BLUE":
brush = this.blueBrush;
break;
default:
brush = this.blackBrush;
break;

string status = "Recognized: " + e.Result.Text + " " +
e.Result.Confidence;
this.ReportSpeechStatus(status);
Dispatcher.BeginInvoke(new Action(() => f tbColor.Background = brush; 1), DispatcherPriority.Normal);

private void RejectSpeech(RecognitionResult result) {
string status = "Rejected: " + (result == null ?
string.Empty : result.Text + " " + result.Confidence);
this.ReportSpeechStatus(status);
Dispatcher.BeginInvoke(new Action(() => f tbColor.Background = blackBrush; 1), DispatcherPriority.Normal);

private void ReportSpeechStatus(string status) {
Dispatcher.BeginInvoke(new Action(() => f tbSpeechStatus.Text = status; 1), DispatcherPriority.Normal);

For additional fine tuning, you could define additional confidence. values for specific terms. This will optimize for false positives as well as terms that are harder to recognize.
KinectExplorer Color Viewer C# How To Overview This How To contains user tasks from the KinectExplorer C# sample. The color viewer portion of the Kinect Explorer C# sample demonstrates how to enable a color stream and display the color stream data. To see all the detail in the KinectExplorer C# sample, use the Kinect for Windows SDK Sample Browser.

Code It Enable the color data stream Use the kinectSensor to enable the color data stream, start streaming data and register for event notifications.
KinectSensor kinectSensor;
kinectSensor.ColorStream.Enable(ColorImageFormat.RgbResolution64 Ox480Fps30);
kinectScnsor.Start();
kinectSensor.ColorFrameReady += ColorImageReady;
Call the Enable method to initialize the image data format, the framerate, and the resolution. Valid values are in the ColorImageFormat enumeration. The default value is RgbResolution640x480Fps30.
Once initialized, start the data stream by calling the Start method.
When the sensor has the next color frame ready, the ColorFrameReady event notifies the application. Register for notification from this event by adding a ColorImageReady handler. You can choose to receive notification of the next frame of color data being ready or the AllFramesReady event which happens when the next frame of all streams is ready (color, depth, and skeleton if they are all enabled).
Get the color data First, declare a member variable to store the pixel data and allocate the memory. Get the number of bytes of memory to allocate from the PixelDataLength property.
private byte[] pixelData;
this.pixelData = new byte[imageFrame.PixelDataLength];
For best performance, only allocate the memory once outside of the render loop that runs every frame.
The system notifies your application that a color frame is ready by firing the ColorFrameReady event. Use this event handler to copy the pixel data from the sensor into the pixelData member.
private void ColorImageReady(object sender, ColorImageFrameReadyEventArgs e) using (ColorImageFrame imageFrame =
e.OpenColorImageFrame()) if (imageFrame != null) imageFrame.CopyPixelDataTo(this.pixelData);
else // imageFrame is null because the request did not arrive in time Use the OpenColorImageFrame method to access the next frame of data that is returned from the sensor. Once you have access to the frame, use the CopyPixelDataTo method to copy the data to your local memory. You will need to run this code for every color frame that arrives from the sensor that you want to save, so be sure to only include in this loop processing that must be done every frame.
The else case may occur is that your application does not respond quickly enough to the event notification. If that is the case, the imageFrame parameter will be null which means you no longer have access to this frame of data as the sensor has now begun to generate the next frame of data and this frame is no longer valid.
Render the color data Create a bitmap to store the new pixel data.
this.outputImage = new WriteableBitmap(imageFrame.Width, imageFrame. Height, 96, 96, PixelFormats.Bgr32, null);
this.kinectColorImage.Source = this.outputImage;
Create a WriteableBitmap, which is a Windows Presentation Foundation (WPF) construct, available in .Net Framework 4, that allows access to the bitmap bits. Instead of creating a Bitmap every frame to save the pixel data, you will get better performance by creating a WriteableBitmap only when the pixel format changes.
Write the sensor pixel data to the WriteableBitmap.
this.outputImage.WritePixels(new Int32Rect(0, 0, imageFrame.Width, imageFrame.Height), this.pixelData, imageFrame.Width * Bgr32BytesPerPixel, 0);
Use the WriteableBitmap.WritePixels method to save the pixel data. You will need to run this code for every frame that arrives from the sensor so be sure to do as little work inside this loop as you can to get the best performance.

KinectExplorer Skeleton C# How To Overview This How To contains user tasks from the KinectExplorer C# sample. The skeleton portion of the Kinect Explorer C# sample demonstrates how to render the skeleton stream, which tracks the people detected within a frame. A skeleton is displayed only if the entire body of at least one person fits within the captured frame. To see all the detail in the KinectExplorer C# sample, use the Kinect for Windows SDK Sample Browser.
Code It Enable the skeleton data stream Use the kinectSensor to enable the skeleton data stream, start streaming data and register for event notifications.
KinectSensor kinectSensor;
kinectSensor.SkeletonStream.Enable();
kinectSensor.Start();
kinectSensor.AllFramesReady += new EventHandier<A11FramesReadyEventArgs>(KinectAllFramesReady);
Call the Enable method to enable the sensor to stream skeleton data.
Once initialized, start the data stream by calling the Start method.
Use the AllFramesReady event to notify the application when the next frame of skeleton data is available. This event notifies the application that the next frame of all streams is ready(color, depth, and skeleton if they are all enabled).
Get the skeleton data First, declare a member variable to store the pixel data and allocate the memory. Get the number of bytes of memory to allocate from the SkeletonArrayLength property.
private Skeleton[] skeletonData;
public SkeletonFrame skeletonFrame this.skeletonData = new Skeleton[kinect.SkeletonStream.FrameSkeletonArrayLength];
This example allocates skeletonData to point to an array of an array of Skeletons withkinectSkeletonStream.FrameSkeletonArraylength elements in it For best performance, only allocate the memory once outside of the render loop that runs every frame.
The system notifies your application that a skeleton frame is ready by firing the AllFramesReady event. Use this event handler to copy the pixel data from the sensor into the skeletonData member.
private void KinectAllFramesReady(object sender, AllFramesReadyEventArgs e) using (SkeletonFrame skeletonFrame =
e.OpenSkeletonFrame()) if (imageFrame != null) imageFrame.CopySkeletonDataTo(this.skeletonData);

else // imageFrame is null because the request did not arrive in time 1 }
Use the OpenSkeletonFrame method to access the next frame of data that is returned from the sensor. Once you have access to the frame, use the CopySkeletonDataTo method to copy the data to your local memory. You will need to run this code for every color frame that arrives from the sensor that you want to save, so be sure to only include in this loop processing that must be done every frame.
The else case may occur is that your application does not respond quickly enough to the event notification. If that is the case, the imageFrame parameter will be null which means you no longer have access to this frame of data as the sensor has now begun to generate the next frame of data and this frame is no longer valid.
Ways to use skeleton data Some uses for depth data include the following:
= Understand presence and location of people = Watch data over time for to detect movement and gestures = Map to depth or color spaces = Store the skeleton data = Use the skeleton data together with color and depth data See what you can im Kinect Explorer C# Sample Overview The Kinect Explorer sample demonstrates Kinect NUI processing by capturing depth stream, color stream and skeletal tracking frames and displaying them on the screen.
The sample also shows how to locate an audio source using beam forming. This block diagram helps you visualize how to sample is divided into functional blocks.
Each of the functional blocks is represented by a different .cs file in the project.
KinectExplorer Depth Viewer C# How To Overview This How To contains user tasks from the KinectExplorer C# sample. The depth viewer portion of the Kinect Explorer C# sample displays the depth stream, showing background objects in grayscale and different people in different colors.
Darker colors in the depth viewer indicate a greater distance from the Kinect sensor. People are color coded in the depth viewer image only if their entire skeleton fits within the captured frame. To see all the detail in the KinectExplorer C# sample, use the Kinect for Windows SDK Sample Browser.
Code It Enable the depth data stream Use the kinectSensor to enable the color data stream, start streaming data and register for event notifications.
KinectSensor kinectSensor;
kinectSensor.DepthStream.Enable(DepthImageFormat.Resolution320x2 40Fps30);
kinectSensor.Start();
kinectSensor.DepthFrameReady += new EventHandler<DepthImageFrameReadyEventArgs>(DepthImageReady);
Call the Enable method to initialize the image data format, the framerate, and the resolution. Valid values are in the DepthImageFormat enumeration. The default value is RgbResolution640x480Fps30.
Once initialized, start the data stream by calling the Start method.
When the sensor has the next depth frame ready, the DepthFrameReady event notifies the application. Register for notification from this event by adding a DepthImageReady handler. You can choose to receive notification of the next frame of depth data being ready or the AllFramesReady event which happens when the next frame of all streams is ready (color, depth, and skeleton if they are all enabled).

Get the depth data First, declare a member variable to store the pixel data and allocate the memory. Get the number of bytes of memory to allocate from the PixelDataLength property.
private short[] pixelData;
private byte[] depthFrame32;
pixelData ¨ new short[imageFrame.PixelDataLength];
depthFrame32 = new byte[imageFrame.Width * imageFrame.Height *
4];
For best performance, only allocate the memory once outside of the render loop that runs every frame.
The system notifies your application that a depth frame is ready by firing the DepthFrameReady event. Use this event handler to copy the pixel data from the sensor into the pixelData member.
private void DepthImageReady(object sender, DepthImageFrameReadyEventArgs e) using (DepthImageFrame imageFrame =
e.OpenDepthImageFrame()) if (imageFrame != null) imageFrame.CopyPixelDataTo(this.pixelData);

else // imageFrame is null because the request did not arrive in time Use the OpenDepthImageFrame method to access the next frame of data that is returned from the sensor. Once you have access to the frame, use the CopyPixelDataTo method to copy the data to your local memory. You will need to run this code for every color frame that arrives from the sensor that you want to save, so be sure to only include in this loop processing that must be done every frame.
The else case may occur is that your application does not respond quickly enough to the event notification. If that is the case, the imageFrame parameter will be null which means you no longer have access to this frame of data as the sensor has now begun to generate the next frame of data and this frame is no longer valid.

Convert the depth data Convert the depth data based on the whether the sensor gather depth and player or just depth data.
byte[] convertedDepthBits =
this.ConvertDepthFrame(this.pixelData, ((KinectSensor)sender).DepthStream);
This function converts a 16-bit grayscale depth frame which includes player indexes into a 32-bit depth frame that displays different players in different colors. For more detail, see the KinectDepthViewer.xaml.cs file in the KinectExplorer sample.
Render the depth data Create a bitmap to store the new pixel data.
KinectDepthViewer kinectDepthImage;
WriteableBitmap outputImage;
this.outputImage new WriteableBitmap(imageFrame.Width, imageFrame.Height, 96, 96, PixelFormats.Bgr32, null);
this.kinectDepthImage.Source = this.outputImage;
Create a WriteableBitmap, which is a Windows Presentation Foundation (WPF) construct, available in .Net Framework 4, that allows access to the bitmap bits. Instead of creating a Bitmap every frame to save the pixel data, you will get better performance by creating a WriteableBitmap only when the pixel format changes.
Write the sensor pixel data to the WriteableBitmap.
this.outputImage.WritePixels(new Int32Rect(0, 0, imageFrame.Width, imageFrame.Height), this.pixelData, imageFrame.Width * Bgr32BytesPerPixel, 0);
Use the WriteableBitmap.WritePixels method to save the pixel data. You will need to run this code for every frame that arrives from the sensor so be sure to do as little work inside this loop as you can to get the best performance.
Ways to use depth data Here are some ways to use depth data:
= Render the depth data = Store the depth data = Render the depth data based on the color data and/or the skeleton data See what you can i Functions This topic has not yet been rated - Rate this topic Describes NUI functions.
In This Section HasSkeletalEngine Determines whether the sensor instance has a skeletal engine.
NuiCameraElevationGetAngle Gets the elevation angle of the Kinect sensor.
NuiCameraElevationSetAngle Sets the elevation angle of the Kinect sensor.
NuiCreateSensorByIndex Creates an instance of the Kinect sensor with the specified index so that an application can open and use it.
NuiCreateSensorById Creates an instance of the Kinect sensor with the specified ID so that an application can open and use it.
NuiDepthPixelToDepth Unpacks the depth value from the packed pixel format.
NuiDepthPixelToPlayerIndex Unpacks the player index value from the packed pixel format.
NuiGetAudioSource Gets the audio source for the first Kinect sensor found.
NuiGetSensorCount Returns the number of Kinect sensors that are connected to the computer.
NuiImageGetColorPixelCoordinatesFromDepthPixel Gets the pixel coordinates in color space that correspond to the specified pixel coordinates in depth space.
NuiImageGetColorPixelCoordinatesFromDepthPixelAtResolution Gets the pixel coordinates in color space that correspond to the specified pixel coordinates in depth space, using the specified depth resolution.
NuiImageResolutionToSize Converts the resolution to a size.
NuamageStreamGetImageFrameFlags Gets the image frame flags for the specified stream.
NuiImageStreamGetNextFrame Gets the next frame of data from the specified image stream.
NuiImageStreamOpen Opens an image stream.

NuiImageStreamReleaseFrame Releases the specified frame of data from the specified stream.
NuarnageStreamSetImageFrameFlags Sets the image frame flags for the specified stream.
NuiInitialize Initializes the NUI engine.
NuiSetDeviceStatusCallback Sets a callback function that gets notified when the sensor connection status changes.
NuiSetFrameEndEvent Sets the event that signals the last frame.
NuiShutdown Shuts down the NUI engine.
NuiSkeletonGetNextFrame Gets the next frame of data from the skeleton stream.
NuiSkeletonSetTrackedSkeletons Sets an array of IDs for skeletal tracking.
NuiSkeletonTrackingDisable Disables skeleton tracking.
NuiSkeletonTrackingEnable Enables skeleton tracking.
NuiTransformDepthImageToSkeleton(LONG, LONG, USHORT) Returns the skeleton space coordinates of a given pixel in the depth image.
NuiTransformDepthImageToSkeleton(LONG, LONG, USHORT, NUI IMAGE RESOLUTION) Returns the skeleton space coordinates of a given pixel in the depth image.
NuiTransformSkeletonToDepthImage(Vector4, FLOAT*, FLOAT*) Returns the depth space coordinates of the specified point in skeleton space.
NuiTransformSkeletonToDepthImage(Vector4, FLOAT*, FLOAT*, NUI IMAGE RESOLUTION) Returns the depth space coordinates and resolution of the specified point in skeleton space.
NuiTransformSkeletonToDepthImage(Vector4, LONG*, LONG*, USHORT*) Returns the depth space coordinates and depth of the specified point in skeleton space.
NuiTransformSkeletonToDepthImage(Vector4, LONG*, LONG*, USHORT*, NUI IMAGE RESOLUTION) Returns the depth space coordinates, depth, and resolution of the specified point in skeleton space.
NuiTransformSmooth Filters skeleton positions to reduce jitter between frames.
XMVectorFromVector4 Converts data from a Vector4 to an XMVector format.
XMVectorToVector4 Converts data from an XMVector to a Vector4 format.

NuiTransformSkeletonToDepthImage(Vector4, FLOAT*, FLOAT*) Returns the depth space coordinates of the specified point in skeleton space.
Syntax C++
void NuiTransformSkeletonToDepthImage( Vector4 vPoint, FLOAT *pfDepthX, FLOAT *pfDepthY
Parameters vPoint Type: Vector4 [in] The point to transform, in skeleton space coordinates. This point must have a valid depth value.
pfDepthX
Type: FLOAT
[out] A pointer to a FLOAT that receives the X coordinate of the point in depth space. This pointer must be non-NULL when you call the function. If this is a NULL pointer, this function returns without doing anything.
pfDepthY
Type: FLOAT
[out] A pointer to a FLOAT that receives the Y coordinate of the point in depth space. This pointer must be non-NULL when you call the function. If this is a NULL pointer, this function returns without doing anything.
Return Value None.
Remarks You can use this function to place virtual objects into the color image with depth-based occlusion. Call this function to determine which skeleton-space vertices of a virtual object are in front of, or behind, real objects that are visible in the depth map image.
After occluded parts of the virtual objects have been culled, use the NuiImageGetColorPixelCoordinatesFromDepthPixel function to overlay the virtual object's vertices into color image pixel coordinates for rendering.
Note The output for this function is computed by applying a transform based on the optics of the depth camera. If the input point is not visible to the depth camera, the output coordinates will be outside the bounds of the depth image, and the outputted data will be invalid.
This function is defined as an inline function in NuiSkeleton.h.

Requirements Header: Declared in NuiSkeleton.h, however, include NuiApi.h in your project.
NuiImageStreamSetImageFrameFlags Sets the image frame flags for the specified stream.
Syntax C++
HRESULT NuiImageStreamSetImageFrameFlags( HANDLE hStream, DWORD dwImageFrameFlags Parameters hStream Type: HANDLE
[in] A handle to the stream.
dwImageFrameFlags Type: DWORD
[in] The image frame flags.
Return Value Type: HRESULT
Returns S_OK if successful; otherwise, returns one of the following failure codes:
E_INVALIDARG, E_NUI_DEVICE_NOT_READY.
Remarks _ The maximum number of output frames you can set is defined by NUIJMAGE_STREAM_FRAME_LIMIT_MAXIMUM.
Requirements Header: Declared in NuiImageCamera.h, however, include NuiApi.h in your project.
Did you find Structures Describes NUI structures.
In This Section NUI IMAGE FRAME Structure Contains information about a depth or color image frame.
NUI IMAGE VIEW AREA Structure This structure is not used. Specifies zoom and panning settings for color still images.
NUI LOCKED RECT Structure Defines the surface for a locked rectangle.

NUI SKELETON DATA Structure Contains the data for one skeleton, including overall position, skeleton joint positions, and whether each skeleton joint is tracked successfully.
NUI SKELETON FRAME Structure Contains information about a frame of data from the skeleton pipeline.
NUI SURFACE DESC Structure Surface image size.
NUI TRANSFORM SMOOTH PARAMETERS Structure Skeleton data smoothing parameters.
Vector4 Structure Represents a vector of four 32-bit floating-point or integer components that are optimally aligned.
Enumerations Describes NUI enumerations.
In This Section NUI IMAGE DIGITALZOOM Enumeration Digital zoom options.
NUI IMAGE RESOLUTION Enumeration Resolution options.
NUI IMAGE TYPE Enumeration Data type options.
NUI SKELETON POSITION INDEX Enumeration Skeleton position options.
NUI SKELETON POSITION TRACKING STATE Enumeration Skeleton position tracking options.
NUI SKELETON TRACKING STATE Enumeration Skeleton tracking options.
Kinect Audio Overview About the Kinect Microphone Array Kinect audio features are supported by a microphone array. In general, a microphone array consists of a set of microphones¨typically four¨that are usually several centimeters apart and arranged in a linear or L-shaped pattern. An array of multiple independent microphones has the following significant advantages over a single microphone:
= Improved audio quality Microphone arrays can support more sophisticated and effective noise suppression and acoustic echo cancellation (AEC) algorithms than are possible with a single microphone.
= Beamforming and source localization By using the fact that the sound from a particular audio source arrives at each microphone in the array at a slightly different time, beamforming allows applications to determine the direction of the audio source and use the microphone array as a steerable directional microphone.

For a detailed discussion of microphone arrays and microphone array support in Windows, see the white papers Microphone Array Support in Windows Vista and How to Build and Use Microphone Arrays for Windows Vista on the MSDN website. For a general conceptual discussion, see the Beamforming topic on Wikipedia.
The Kinect sensor includes a four-element linear microphone array, which uses 24-bit ADC and provides local signal processing, including acoustic echo cancellation and noise suppression. Applications that are created with this SDK can use the Kinect microphone array for the following:
= High-quality audio capture.
= Beamforming and source localization. The KinectAudio DMO includes built-in algorithms that control the "beam" and provide the source direction to applications.
= Speech recognition.
With this SDK, applications can use the Kinect microphone array as an input device for the Microsoft.Speech speech recognition API.
Use either managed or unmanaged APIs for audio capture and control, as described in the following sections.
C++ Audio API
Raw Audio Capture Applications can use the WASAPI to capture the raw audio stream from the Kinect sensor's microphones. For examples of the steps to capture a raw audio stream, see AudioCaptureRaw C++ How To.
KinectAudio DMO
Windows Vista and later versions include a voice-capture digital signal processor (DSP) that supports microphone arrays. Developers typically access that DSP
through a DMO, which is a standard COM object that can be incorporated into a DirectShow graph or a Microsoft Media Foundation topology. The SDK includes an extended version of the Windows microphone array DMO¨referred to here as the KinectAudio DMO¨to support the Kinect microphone array.
Although the internal details are different, the KinectAudio DMO supports the same interfaces as the standard microphone array DMO and works in much the same way.
However, the KinectAudio DMO:
= Supports an additional microphone mode, which is customized to support the Kinect microphone array.

= Includes beamforming and source localization functionality, which is exposed through theINuiAudioBeam Interface.
= Get the DMO by calling NuiGetAudioSource or INuiSensor:NuiGetAudioSource Method.
The beamforming functionality supports 11 fixed beams, which range from -50 to +50 degrees in 10 degree increments. Applications can use the DMO's adaptive beamforming option¨which automatically selects the optimal beam¨or specify a particular beam. The DMO also includes a source localization algorithm, which estimates the source direction.
This SDK includes the following two samples that show how to use the KinectAudio DMO to capture an audio stream from the Kinect sensor's microphone array:
= MicArrayEchoCancellation shows how to capture sound and determine the selected beam and source direction by using the DMO as a DirectShow source.
For examples of the user tasks required see the MicArrayEchoCancellation C++ How To.
= MFAudioFilter shows how to capture sound by using the KinectAudio DMO as a filter object in a Media Foundation topology. For examples of the user tasks required see the MFAudioFilter C++ How To.
To access the samples, see the Kinect for Windows SDK Sample Browser.
C# Audio API
The SDK includes a managed Audio API, which is basically a wrapper over the KinectAudio DMO that supports the same functionality but is much simpler to use. The managed API allows applications to configure the DMO and perform operations such as starting, capturing, and stopping the audio stream. The managed API also includes events that provide the source and beam directions to the application.
The KinectAudioDemo sample shows how to use the managed API to monitor the audio source direction and how to perform speech recognition on the kinect audio stream.
The RecordAudio sample shows how to use the managed API to capture sound and monitor the selected beam and source direction in a console application. For examples of the user tasks required see the Audio Demo C# How To.
To access the samples, see the Kinect for Windows SDK Sample Browser.
How to Use Microsoft.Speech with the Kinect Microphone Array Speech recognition is a key aspect of natural user interface. The SDK provides the necessary infrastructure for managed applications to use the Kinect microphone with the Microsoft.Speech API, which supports the latest acoustical algorithms.
This SDK

includes a custom acoustical model that is optimized for the Kinect sensor's microphone array.
MSDN provides limited documentation for the Microsoft.Speech API. Instead, install the Microsoft Speech Platform - Software Development Kit (SDK) and use the installed HTML Help file (.chm) which can be found at Program Files\Microsoft Speech Platform SDK\Docs (see the software requirements to download the SDK).
Note You can use the Kinect microphone array with the System.Speech API. In particular, Microsoft .Speech does not support dictation, so you must use System .Speech for that purpose. However, you cannot use the Kinect for Windows Runtime Language Pack with System .Speech. Instead, use the standard Windows language pack, which is not optimized for the Kinect microphone array and might not provide the same level of recognition accuracy.