Chapter 5

SOCIAL INTERACTION
 Overview
• What is meant by social interaction
• The social mechanisms used in conversations
• What is meant by social presence
• Overview of technologies for supporting social
interaction, social presence and social games
• Illustrate how video conferencing and other
social media platforms evolved during COVID-19
• How technology can augment people who are
co-present
www.id-book.com 2
 Social interaction
• We live together, work together, play
together, talk to each other, and socialize
• Social technologies developed to enable
us to persist in being social when apart
 They differ in how they support us
 Some encourage social interactions (for
example, family games with Alexa)
 Others have a negative impact on everyday
conversations (Turkle, 2015)

www.id-book.com 3
Are we spending too much time in our
own digital bubbles?

www.id-book.com 4
 Questions raised by social tech
• Are in person conversations being superseded by
social media interactions?
• How many friends do you have on Facebook,
LinkedIn, WhatsApp and so on versus real life?
• How much do they overlap?
• How are the ways that we live and interact with one
another changing?
• Are the established rules and etiquette still
applicable to online and offline?

www.id-book.com 5
 Conversational mechanisms
Various mechanisms and ‘rules’ are followed
when holding a conversation face to face,
such as mutual greetings
A: Hi there
B: Hi!
C: Hi
A: All right?
C: Good, how’s it going?
A: Fine, how are you?
C: OK
B: So-so. How’s life treating you?

www.id-book.com 6
 Conversational rules
Sacks et al. (1978) conversation analysis of
conversations propose three basic rules:

Rule 1: The current speaker chooses the next speaker

by asking an opinion, question, or request

Rule 2: Another person decides to start speaking

Rule 3: The current speaker continues talking

www.id-book.com 7
 More conversational rules
Turn-taking used to coordinate conversation
A: Shall we meet at 8:00?
B: Um, can we meet a bit later?
A: Shall we meet at 8:00?
B: Wow, look at him?
A: Yes what a funny hairdo!
B: Um, can we meet a bit later?
Back channeling to signal to continue and
following
 Uh-uh, umm, ahh

www.id-book.com 8
 Further conversational rules
Farewell rituals
• Bye then, see you, yeah bye, see you later….

Implicit and explicit cues

• For instance, looking at watch or fidgeting with coat
and bags
• Explicitly saying, “Oh dear, look at the time, I must go,
I’m running late…”

www.id-book.com 9
 Breakdowns in conversation
When someone says something that
is misunderstood:
 Speaker will repeat with emphasis:
A: “This one?”
B: “No, I meant that one!”

 Also use tokens:

Eh? Quoi? Huh? What?

www.id-book.com 10
 What happens in online
conversations?
• Do the same conversational rules apply?
• Are there different kinds of breakdowns?
• How do people repair them for:
 Email?
 Instant messaging?
 Texting?
 Skype or other videoconferencing software?

www.id-book.com 11
New social rules during COVID-19
• For how to behave and interact with others
when communicating via video conferencing
• Muting yourself when not talking
• Raising a yellow hand when wanting to speak
• Other emoji reactions became commonly
used to signify various forms of praise,
emotions and non-verbal feedback, e.g.
– clicking on the red heart and party hat icons that
momentarily appear in someone’s window and
then disappear after 10 seconds

www.id-book.com 12
 Online collaboration and
communication
• Much research on how to support
conversations when people are remote
• Many applications have been developed
 email, videoconferencing, instant messaging,
chatrooms

• To what extent do they mimic or move

beyond existing ways of conversing?

www.id-book.com 13
 Talking with Chatbots
• Conversational User
Interfaces, such as
chatbots, are more
sophisticated in how they
emulate turn-taking in
conversations
– e.g. Replika
• Recent developments in
AI are changing how we
converse and interact
with AI
– e.g, ChatGTP

www.id-book.com 14
 Dilemma: Is it OK to talk with a
dead person using a chatbot?
• Eugenia Kuyda lost a close friend in a car
accident who was only in his 20s
• She took all his texts sent over the course of
his life and made a chatbot using them
• Chatbot responds to text messages so that
Eugenia can talk to her friend as if he was
alive
• Is this a creepy or comforting way to deal with
grief?
 Is it respectful of the dead person?
www.id-book.com 15
 Early videoconferencing:
VideoWindow (Bellcore, 1989)
• Shared space that allowed people 50
miles apart to carry on a conversation
as if in same room drinking coffee
together
• 3 x 8 foot ‘picture-window’ between two
sites with video and audio
• People did interact via the window, but
strange things happened (Kraut, 1990)
www.id-book.com 16
Diagram of VideoWindow in use

www.id-book.com 17
 Findings of how VideoWindow
System was used

• Talked constantly about the system

• Spoke more to other people in the same
room rather than in other room
• When trying to get closer to someone in the
other place it had opposite
effect participants went out of range of the
camera and microphone
• No way of monitoring this
www.id-book.com 18
 Videoconferencing and
telepresence rooms
• Many to choose from to connect multiple people
(e.g. Zoom, Teams, Google Meet)
• Customized telepresence rooms for groups

www.id-book.com 19
 Current videoconferencing
• Teams, Zoom etc., have greatly extended how we
communicate while providing tools to make is
easier to switch between talking and working
together
• Zoom fatigue came into being (Bailenson, 2021)
– excessive amounts of close-up eye gaze
– intense cognitive load
– increased self-evaluation from staring at a video of
oneself
– physically being in the same place for hours on end

www.id-book.com 20
Microsoft prototype of a technology-
enhanced hybrid meeting

www.id-book.com 21
 The Metaverse

• Meta’s vision of three friends socializing in a 3D world

represented as torso avatars

• Users experience each other through donning VR headsets

www.id-book.com 22
 Telepresence and social presence
• Telepresence refers to one party being
present with another party, who is present
in a physical space, such as a meeting
room
• Social presence refers to the feeling of
being there with a real person when in
virtual reality

www.id-book.com 23
 How much realism and immersion
are necessary..?
• …in telepresence to make it compelling?
• Telepresence rooms try to make remote
people appear to be life-like
 Use multiple high-definition cameras with eye-
tracking features and directional microphones

• Does FaceTime have as much presence

as more high-definition settings?

www.id-book.com 24
The future: 3D person in a box?

David Nussbaum demonstrating how they capture and

www.id-book.com 25
present the Proto person in a box
 Proto M

Talking with a 3D video of granny in a box. The embedded camera at the top of the box
faces the mother and child so that Granny can see and hear them in real time.
www.id-book.com 26
 Online collaborative tools

• Now more commonplace in our

everyday and working lives
– e.g. the sharing of
calendars, word
processing, design and
project management tools
like Slack
• Places for sharing knowledge
• Tools like Miro, Overleaf and
Google Docs enable online
collaborative creation and
editing of reports, designs,
etc.,

www.id-book.com 27
 Collaborative online tools

Screen shot of a Miro board used in an online class

www.id-book.com 28
on Interaction Design where students upload their posters
 What is co-presence?
• Co-located groups
who want to
collaborate
• Many technologies
have been designed
to:
 Enable groups to work,
learn and socialize more
effectively together
 For example, tabletops,
whiteboards, and public
displays

www.id-book.com 29
 Coordination mechanisms
• When a group of people act or interact together,
they need to coordinate themselves
 For example, when playing football or navigating a
ship

• To do so, they use:

 Verbal and non-verbal communication
 Schedules, rules, and conventions
 Shared external representations

www.id-book.com 30
In person coordinating mechanisms
• Talk is central

• Non-verbal also used to emphasize and as a

substitute
 e.g., nods, shakes, winks, glances, gestures, and
hand-raising

• Formal meetings
 Explicit structures such as agendas, memos, and
minutes are employed to coordinate the activity

www.id-book.com 31
 Awareness mechanisms
Involves knowing who is around, what is happening, and who
is talking with whom (Dourish and Bly, 1992)

Peripheral awareness
• Keeping an eye on things happening in the periphery of vision
• Overhearing and overseeing—allows tracking of what others are doing
without explicit cues

Situational awareness
• Being aware of what is happening around you in order to understand how
information and your actions will affect ongoing and future events
• For example, air traffic control or an operating theatre

www.id-book.com 32
 Sharable interfaces

Designed to capitalize on existing forms of

coordination and awareness mechanisms

Several studies investigating whether they help

people to work together better, have found:
• More equitable participation
• More natural to work around
• More comfortable sitting around a table than standing in
front of a display

www.id-book.com 33
 The Reflect Table
• LEDs lit up to reflect how
much each member of
the group spoke
• Used microphones in
front of each individual to
do this
• Study showed those who
spoke the most changed
their behavior the most
• Those who spoke the
least did not change their
behavior
• Why do you think this is?
www.id-book.com 34
Playing together in same space
• Visitors using an AR
sandbox at the V and A
• Visitors sculpt
landscapes out of sand
• System reacts with
changing superimposed
digital colored
landscape
• Enables creative forms
of collaboration

www.id-book.com 35
PeopleLens: A head mounted device
that enhances a blind child’s spatial
awareness of those around them

www.id-book.com 36
 Social Games

Designed to facilitate social interaction

can be played indoors examples include board

games, tabletop games,
or outdoors, with and and videogames (such as
without technology. Minecraft)

Each player is aware of other players’

presence, their actions and how well they are
playing

www.id-book.com 37
 Social online games
• Can involve creating a community, where competition,
collaboration, peer pressure, rebellion, jealousy, and so on are all
played out in their various forms
• Matt Richetti (2022) has proposed three heuristics for evaluating
them:
– Does the social game involve synchronous or asynchronous player interaction, where
players either chat with each other or they take turns.

– Is the social interaction symmetrical or asymmetrical, in the sense that does forming a
relationship require input from both parties or can they be formed unilaterally by a
single party?

– Does the social relationship involve strong or weak ties in the sense of whether the
relationships between players become deep and long lasting or are they transitory?
• By asking these questions, game designers can think
about the kinds of social interactions they want to
support

www.id-book.com 38
 Summary
Social interaction is central to our everyday lives

Social mechanisms, like turn-taking, enable us to collaborate and coordinate our activities

Keeping aware of what others are doing and letting others know what you are doing are
important aspects of collaborative working and socializing

Many technologies have been built to support telepresence, social presence, and co-presence

Social media has brought about significant changes in how people keep in touch and manage
their social lives

www.id-book.com 39