Pervasive & Ubiquitous Computing

(CoSc 6304)
5. User Interface for Ubiquitous
Computing
 Introduction
User Interface
– Point of Contact Between Computer System and
Human
– Both in terms of Input and Output
– Input : Keyboard, Mouse, Touch Screen, Digital Camera
etc.
– Output : CRT/LCD/LED Displays, Projector etc.
Ubiquitous Computation (Ubicomp)
– Computation Every Where
– Interfaces to Ubicomp Systems must reside in
periphery of our user’s attention and should remain
unnoticed until required.
– a ubicomp system is made up of subsystems and we must design
for the experience not the individual subsystem
 GUI To Context Data
Graphical User Interface (GUI)
–Current GUI is built considering keyboard,
Screen, Mouse devices
–GUI offers intuitive Windows, Icons, Menus,
Pointer rather command line
–Inputs from keyboards, mouse,
Ubicomp User Interface (UUI)
–Must Consider Broader Range of Inputs
–Like human motion, activity, preference and
action desired.
–Must understand Context of action.
 Rules for UUI Design

1. Bliss 6. Modelessness
( easy to learn ) ( Avoid Modes)

2. Distraction 7. Fear of Interaction

( don’t need concentration ) ( easy ways to undo )

3. Cognitive Flow 8. Notifications

( access every where ) ( display feedback )

4. Manuals 9. Calming
(should not needed ) ( human inputs and senses)

5. Transparency 10. Defaults

( context aware ) ( reuse user inputs)
 Interaction Design
–Interaction Design is discipline of defining the
expected behavior of products and systems
that a user can interact with
–UUI design is field of Interaction design
Interaction Design Depends
–Complexity of proposed system
–Its novelty
–Degree of stability or ubiquity
–and Its COST
Design Methodologies for Ubicomp
–User-Centered Design (UCD)
–Systems Design
–Genius Design
 User-Centered Design (UCD)
–Focuses on USER’s Need, Problems, and
Goals
–User is involved at every stage of process
–User needs to ensure system works correctly
–UCD plays important role in Ubicomp research
and Development
Disadvantages
–Evolving stated Vs Actual need
–Technological Shifts
–Simply involving wrong set or type of user in
process
–It does not consider interaction between two
systems
 System Design and Genius Design

System Design
–is a systematic and compositional approach to
development, based on the combination of
components to realize a solution in essence the
development of a system of systems
–Ubicomp System is typically of many systems social
system (people), devices, applications, computational
artifects, sensors, actuators, and services
–Unlike Desktop or Web application software
development Systems Design Must be Context aware
and act accordingly
–A systems design approach forces a designer to
consider the entire environment in which the ubicomp
system will be realized and not just one component of
it.
 Genius Design

–the process of exclusively relying on the

wisdom and experience of the designer to
make all the design decisions.
–Designer Don’t take input from users to design
system , uses own experience
–Most of Apple Products Like iPhone, iPad etc
designed using this approach
 Classes of User Interfaces

• Presence of Input Technologies which do

not cleanly (easily) fit into any of these
Six classes, because they rely on new
devices

• Examples include body movement in the

form of gesture, speech, ambient
feedback, surface interaction, and
augmented reality (AR)
 Classes of User Interfaces

• New Classes of interfaces

– Tangible User Interface
– Surface User Interface
– Ambient User Interface
 Tangible Interfaces
Earlier Known as Graspable User Interface
User interacts with digital information through
physical environment
Unlike a GUI, which presents manipulateable
elements virtually onscreen, a Tangible User
Interface (TUI) integrates both representation and
control of computation into physical artifacts.
Characteristics of tangible user interfaces
– Physical representations are computationally
coupled to underlying digital information.
– Physical representations embody mechanisms for
interactive control.
– Physical representations are perceptually coupled to
actively mediated digital representations.
 Examples

• Tangible Disaster Simulation System

• A collaborative tool for planning disaster
measures based on disaster simulation
and evacuation simulation using
Geographic Information Systems (GIS).
• SandScape
• a tangible interface for and
understanding landscapes
through a variety of
computational simulations
using sand.
 Surface User Interfaces
• An SUI is a class of user interface that relies
on a self-illuminated [e.g., liquid crystal
display (LCD)] or projected horizontal,
vertical, or spherical interactive surface
• coupled with control of computation into the
same physical surface (e.g., a touchscreen).
• The outputs and inputs to an SUI are tightly
coupled.
• They rely on computational techniques
–including computer vision
–capacitive and surface acoustic wave
detection, to determine user input to the
 Surface User Interfaces

• They are often used in public places (kiosks,

ATMs) or small personal devices (PDA,
iPhone) where a separate keyboard and
mouse cannot or should not be used.
• It can Scale from Touch Screen of phone to
Huge Screen for store locator in Mall or
Shopping place
 Examples
• Microsoft Surface
• responds to natural hand gestures and real
world objects.
The surface is capable of
–object recognition,
–object/finger orientation recognition
–and tracking, and is
–multi-touch and is multi-user
 Examples…

Mobile Phone Touch Screen

• is an electronic visual display that can detect
the presence and location
of a touch within the display area.
 Ambient User Interfaces
• Ambient User Interfaces ( AUI ) are very comfortable to
user, negligible user involvement
• Ambient information displays or outputs are intended
to be “ignorable” or “glanceable,” allowing users to
perceive the information presented in the periphery of
their attention, but also to be bring this information
(e.g., social reminders on facebook ) into focus as
required.
• AUI is a class of user interface where the output
elements reside in the periphery of a user’s
awareness, moving to the center of attention only
when appropriate and desirable, and the inputs
come from nonintrusive sensing or inference from
other actions
• Fully realized AUIs as defined are not yet
commonplace in our daily lives.
 Examples

• Power Aware Cords

• Currency Data Fountain
• Ambient Umbrella
 Input Technologies
• A UUI relies on a broader range of inputs and outputs
from the system than the classical GUI, TUI, or even
an SUI.
• Examples of these inputs include

–physiological measurements,
–location,
–identity,
–video, audio, gesture, and
–touch.
 Input technology…

In addition,
–personal/embedded sensors, data
–mining, historical data, inference, and
preferences can all act as inputs to a Ubicomp
System
 Output technologies

–Ambient displays, environmental

updates, actuators, automated actions
and personalized
–behaviors, and multiple audio/video
channels.
–Such outputs are all reliant on our
senses including sight, taste, smell,
touch, hearing, and balance
• some user actions will be interpreted via
the UUI as an input to the system without
the user being explicitly aware of it.

• Likewise, the UUI can provide outputs

that are only intended for the periphery of
the user’s attention (Example Room
temparature can be set depending on
presence of person, current temp. etc)
 Sensor Input

• A sensor is a device that can measure a

physical property from the environment.
• Sensors can reside in the environment or on
the body
• Environmental Sensors to monitor (Mobile
Sensors)
– Traffic
– Air Quality
– Water Quality
– Light pollution
– Temperature
 Input sensor

Physiological Sensors
–Used to collect measurement about a person
they are attached to
–Like heart rate , body temperature
–Blood oxygen level sensor ( Ring Sensor )
Ubicomp System uses these sensed data all
together to function correctly
 Gesture Input
• A gesture is the movement of a part of the body to
express an idea of meaning.
• Typical gestures such as pointing, waving, or nodding
are formed with the hand or the head as appropriate
• gesture recognition is the process of interpreting
human gestures using various inputs and
computational processing
• Gesture Sensing Devices
– 3D DEPTH’
SENSORS
– CAMERAS
– Microsoft Kinect
– RFIDs – Animation Creation
 Speech Input
• Speech recognition is the process of interpreting
human speech using a variety of audio inputs and
computational processing
• In UUI Speech can act as both input and output
• Speech represents a popular view of how humans
will interact with computers as evidenced in
literature and film
• Speech recognition and natural speech output
represent the backbone of natural language and
anthropomorphic interfaces
• Environments such as surgeries, dangerous work
environments, and driving all represent environments
where UUIs with speech as input have been
researched and developed
 Interface Usability Metrics

• Usability is a quality attribute that assesses

how easy user interfaces are to use.
• The word “usability” also refers to methods for
improving ease-of-use during the design
process.
• Usability is defined by five quality
components:
– learnability,
– efficiency,
– memorability,
– errors,
– and satisfaction.
• Kinect Games Highly intuitive interface
• For UUI Following are Usability Metrics
• Conciseness
– Simple actions, Few keystrokes or Few clicks,
– Can be measured by TIME
• Expressiveness
– Combinations of actions gives consistent Results

• Ease
– How much does a user need to learn or recall just to start
using the interface?
• Transparency
– How much does a user need to remember about the state of
his or her problem while using the interface telephone speech
interface versus a GUI?
• Discoverability

– Can the user easily understand interface

functionality
• Invisibility
– How much does the interface make itself known
when it could have inferred, deduced, or waited for
the data required
• Programmability

– Can the application, device, or service be used in

repetitive tasks or can it become a component in a
larger system
Thank you