Chapter 7

Multimedia
Networking

A note on the use of these ppt slides: Computer

We’re making these slides freely available to all (faculty, students, readers).
They’re in PowerPoint form so you see the animations; and can add, modify, Networking: A
and delete slides (including this one) and slide content to suit your needs.
They obviously represent a lot of work on our part. In return for use, we only Top Down
ask the following:
 If you use these slides (e.g., in a class) that you mention their source Approach
(after all, we’d like people to use our book!)
 If you post any slides on a www site, that you note that they are adapted
6th edition
from (or perhaps identical to) our slides, and note our copyright of this Jim Kurose, Keith Ross
material. Addison-Wesley
Thanks and enjoy! JFK/KWR March 2012
All material copyright 1996-2012
J.F Kurose and K.W. Ross, All Rights Reserved

Multmedia Networking 7-1

Multimedia networking: outline
7.1 multimedia networking applications
7.2 streaming stored video
7.3 voice-over-IP
7.4 protocols for real-time conversational
applications
7.5 network support for multimedia

Multmedia Networking 7-2

Multimedia networking: outline
7.1 multimedia networking applications
7.2 streaming stored video
7.3 voice-over-IP
7.4 protocols for real-time conversational
applications
7.5 network support for multimedia

Multmedia Networking 7-3

Multimedia: audio
 analog audio signal
sampled at constant
quantization
rate error
quantized
value of
 telephone: 8,000 analog value

audio signal amplitude

samples/sec analog
signal
 CD music: 44,100
samples/sec
 each sample quantized,
i.e., rounded
time
 e.g., 28=256 possible
sampling rate
quantized values (N sample/sec)

 each quantized value

represented by bits,
e.g., 8 bits for 256
values
Multmedia Networking 7-4
Multimedia: audio
 example: 8,000
samples/sec, 256
quantized values: 64,000 quantization
error
quantized
value of
bps analog value

audio signal amplitude

 receiver converts bits back analog
to analog signal: signal

 some quality reduction

example rates time

 CD: 1.411 Mbps sampling rate

(N sample/sec)
 MP3: 96, 128, 160 kbps
 Internet telephony: 5.3
kbps and up

Multmedia Networking 7-5

 spatial coding example: instead
Multimedia: video of sending N values of same
color (all purple), send only two
values: color value (purple) and
number of repeated values (N)
 video: sequence of
images displayed at ……………………...…
……………………...…
constant rate
 e.g. 24 images/sec
 digital image: array of
pixels
 each pixel
represented by bits frame i
 coding: use redundancy
within and between
images to decrease # bits temporal coding example:
used to encode image instead of sending
complete frame at i+1,
 spatial (within image) send only differences from
frame i
 temporal (from one
image to next) frame i+1
Multmedia Networking 7-6
 spatial coding example: instead
Multimedia: video of sending N values of same
color (all purple), send only two
values: color value (purple) and
number of repeated values (N)
 CBR: (constant bit rate):
video encoding rate fixed ……………………...…
……………………...…
 VBR: (variable bit rate):
video encoding rate
changes as amount of
spatial, temporal coding
changes
 examples:
 MPEG 1 (CD-ROM) 1.5 frame i
Mbps
 MPEG2 (DVD) 3-6
temporal coding example:
Mbps instead of sending
 MPEG4 (often used in complete frame at i+1,
send only differences from
Internet, < 1 Mbps) frame i

frame i+1
Multmedia Networking 7-7
Multimedia networking: 3 application types

 streaming, stored audio, video

 streaming: can begin playout before downloading
entire file
 stored (at server): can transmit faster than
audio/video will be rendered (implies
storing/buffering at client)
 e.g., YouTube, Netflix, Hulu
 conversational voice/video over IP
 interactive nature of human-to-human
conversation limits delay tolerance
 e.g., Skype
 streaming live audio, video
 e.g., live sporting event (futbol)
Multmedia Networking 7-8
Multimedia networking: outline
7.1 multimedia networking applications
7.2 streaming stored video
7.3 voice-over-IP
7.4 protocols for real-time conversational
applications
7.5 network support for multimedia

Multmedia Networking 7-9

Streaming stored video:
Cumulative data

2. video
sent
1. video 3. video received,
recorded network delay played out at client
(e.g., 30 (fixed in this (30 frames/sec) time
frames/sec example)
)
streaming: at this time, client
playing out early part of video,
while server still sending later
part of video

Multmedia Networking 7-10

Streaming stored video: challenges

 continuous playout constraint: once client

playout begins, playback must match original
timing
 … but network delays are variable (jitter),
so will need client-side buffer to match
playout requirements
 other challenges:
 client interactivity: pause, fast-forward,
rewind, jump through video
 video packets may be lost, retransmitted
Multmedia Networking 7-11
 Streaming stored video: revisted
constant bit
rate video client video constant bit
Cumulative data

transmission reception rate video

playout at client
variable
network

buffered
video
delay

client playout time

delay

 client-side buffering and playout delay:

compensate for network-added delay, delay
jitter
Multmedia Networking 7-12
 Client-side buffering, playout
buffer fill level,
Q(t)
variable fill playout rate,
rate, x(t) e.g., CBR r

client application
video server buffer, size B

client

Multmedia Networking 7-13

 Client-side buffering, playout
buffer fill level,
Q(t)
variable fill playout rate,
rate, x(t)  e.g., CBR r

client application
video server buffer, size B

client

1. Initial fill of buffer until playout begins at tp

2. playout begins at tp,
3. buffer fill level varies over time as fill rate x(t)
varies and playout rate r is constant
Multmedia Networking 7-14
 Client-side buffering, playout
buffer fill level,
Q(t)
variable fill playout rate,
rate, x(t) e.g., CBR r

client application
video server buffer, size B

playout buffering: average fill rate (x), playout

rate (r):
x < r: buffer eventually empties (causing freezing of
video playout until buffer again fills)
x > r: buffer will not empty, provided initial playout delay
is large enough to absorb variability in x(t)
 initial playout delay tradeoff: buffer starvation less
likely with larger delay, but larger delay until user
begins watching
Multmedia Networking 7-15
Streaming multimedia: UDP
 server sends at rate appropriate for client
 often: send rate = encoding rate = constant
rate
 transmission rate can be oblivious to
congestion levels
 short playout delay (2-5 seconds) to remove
network jitter
 error recovery: application-level,
timeipermitting
 RTP [RFC 2326]: multimedia payload types
 UDP may not go through firewalls

Multmedia Networking 7-16

Streaming multimedia: HTTP
 multimedia file retrieved via HTTP GET
 send at maximum possible rate under TCP

variable
  
rate, x(t)

video TCP send TCP receive application

file buffer buffer playout buffer
server client

 fill rate fluctuates due to TCP congestion

control, retransmissions (in-order delivery)
 larger playout delay: smooth TCP delivery
rate
 HTTP/TCP passes more easily through
firewalls Multmedia Networking 7-17
Streaming multimedia: DASH
 DASH: Dynamic, Adaptive Streaming over HTTP
 server:
 divides video file into multiple chunks
 each chunk stored, encoded at different rates
 manifest file: provides URLs for different chunks
 client:
 periodically measures server-to-client bandwidth
 consulting manifest, requests one chunk at a time
• chooses maximum coding rate sustainable given
current bandwidth
• can choose different coding rates at different points
in time (depending on available bandwidth at time)

Multmedia Networking 7-18

Streaming multimedia: DASH
 DASH: Dynamic, Adaptive Streaming over
HTTP
 “intelligence” at client: client determines
 when to request chunk (so that buffer starvation, or
overflow does not occur)
 what encoding rate to request (higher quality when
more bandwidth available)
 where to request chunk (can request from URL
server that is “close” to client or has high available
bandwidth)

Multmedia Networking 7-19

Content distribution networks
 challenge: how to stream content (selected
from millions of videos) to hundreds of
thousands of simultaneous users?

 option 1: single, large “mega-server”

 single point of failure
 point of network congestion
 long path to distant clients
 multiple copies of video sent over outgoing link
….quite simply: this solution doesn’t scale

Multmedia Networking 7-20

Content distribution networks
 challenge: how to stream content (selected
from millions of videos) to hundreds of
thousands of simultaneous users?

 option 2: store/serve multiple copies of videos

at multiple geographically distributed sites
(CDN)
 enter deep: push CDN servers deep into many
access networks
• close to users
• used by Akamai, 1700 locations
 bring home: smaller number (10’s) of larger
clusters in POPs near (but not within) access
networks
• used by Limelight

Multmedia Networking 7-21

CDN: “simple” content access scenario
Bob (client) requests video http://netcinema.com/6Y7B23V
video stored in CDN at http://KingCDN.com/NetC6y&B23V

1. Bob gets URL for video

http://netcinema.com/6Y7
B23V 2. resolve
from netcinema.com 2 http://netcinema.com/6Y7B23V
1
web page via Bob’s local DNS
6. request video 5
from 4&5. Resolve
KINGCDN server, http://KingCDN.com/NetC6y&B23
streamed via via KingCDN’s authoritative DNS,
netcinema.com 3. netcinema’s DNS returns
4 which returns IP address of
URLHTTP
KIingCDN
http://KingCDN.com/NetC6y&
3 server with video
B23V

netcinema’s
authorative DNS KingCDN.com KingCDN
authoritative DNS Multmedia Networking 7-22
CDN cluster selection strategy
 challenge: how does CDN DNS select “good”
CDN node to stream to client
 pick CDN node geographically closest to client
 pick CDN node with shortest delay (or min # hops)
to client (CDN nodes periodically ping access
ISPs, reporting results to CDN DNS)
 IP anycast

 alternative: let client decide - give client a list

of several CDN servers
 client pings servers, picks “best”
 Netflix approach

Multmedia Networking 7-23

Case study: Netflix
 30% downstream US traffic in 2011
 owns very little infrastructure, uses 3rd party
services:
 own registration, payment servers
 Amazon (3rd party) cloud services:
• Netflix uploads studio master to Amazon cloud
• create multiple version of movie (different
endodings) in cloud
• upload versions from cloud to CDNs
• Cloud hosts Netflix web pages for user
browsing
 three 3rd party CDNs host/stream Netflix
content: Akamai, Limelight, Level-3

Multmedia Networking 7-24

 Case study: Netflix
upload copies of
Amazon
multiple versions
cloud Akamai
of video to CDNs
CDN
Netflix
registration,
3. Manifest file
accounting
servers 2. Bob browses returned for
requested Limelight
Netflix video 2
3 video CDN
1

1. Bob manages
Netflix account
Level-3 CDN
4. DASH
streaming

Multmedia Networking 7-25

Multimedia networking: outline
7.1 multimedia networking applications
7.2 streaming stored video
7.3 voice-over-IP
7.4 protocols for real-time conversational
applications
7.5 network support for multimedia

Multmedia Networking 7-26

Voice-over-IP (VoIP)
 VoIP end-end-delay requirement: needed to
maintain “conversational” aspect
 higher delays noticeable, impair interactivity
 < 150 msec: good
 > 400 msec bad
 includes application-level (packetization,playout),
network delays
 session initialization: how does callee
advertise IP address, port number, encoding
algorithms?
 value-added services: call forwarding,
screening, recording
 emergency services: 911
Multmedia Networking 7-27
VoIP characteristics
 speaker’s audio: alternating talk spurts, silent
periods.
 64 kbps during talk spurt
 pkts generated only during talk spurts
 20 msec chunks at 8 Kbytes/sec: 160 bytes of data
 application-layer header added to each chunk
 chunk+header encapsulated into UDP or TCP
segment
 application sends segment into socket every
20 msec during talkspurt

Multmedia Networking 7-28

VoIP: packet loss, delay
 network loss: IP datagram lost due to network
congestion (router buffer overflow)
 delay loss: IP datagram arrives too late for
playout at receiver
 delays: processing, queueing in network; end-
system (sender, receiver) delays
 typical maximum tolerable delay: 400 ms
 loss tolerance: depending on voice encoding,
loss concealment, packet loss rates between
1% and 10% can be tolerated

Multmedia Networking 7-29

 Delay jitter
constant bit
rate client constant bit
Cumulative data

transmission reception rate playout

at client
variable
network

buffered
data
delay
(jitter)

client playout time

delay

 end-to-end delays of two consecutive

packets: difference can be more or less than
20 msec (transmission time difference)
Multmedia Networking 7-30
VoIP: fixed playout delay
 receiver attempts to playout each chunk
exactly q msecs after chunk was generated.
 chunk has time stamp t: play out chunk at
t+q
 chunk arrives after t+q: data arrives too late
for playout: data “lost”
 tradeoff in choosing q:
 large q: less packet loss
 small q: better interactive experience

Multmedia Networking 7-31

VoIP: fixed playout delay
 sender generates packets every 20 msec during talk spurt.
 first packet received at time r
 first playout schedule: begins at p
 second playout schedule: begins at p’
packets

packets loss
generated
packets
playout schedule
received
p' - r

playout schedule
p-r

time

r
Multmedia Networking 5-32
p p'
Adaptive playout delay (1)
 goal: low playout delay, low late loss rate
 approach: adaptive playout delay adjustment:
 estimate network delay, adjust playout delay at
beginning of each talk spurt
 silent periods compressed and elongated
 chunks still played out every 20 msec during talk
spurt
 adaptively estimate packet delay: (EWMA -
exponentially weighted moving average, recall TCP
RTT estimate):
di = (1)di-1 +  (ri – ti)

delay small time time sent

estimate after constant, received - (timestamp)
ith packet e.g. 0.1
measured delay of ith
packet Multmedia Networking 7-33
Adaptive playout delay (2)
 also useful to estimate average deviation of delay, vi :

vi = (1)vi-1 + |ri – ti – di|

 estimates di, vi calculated for every received
packet, but used only at start of talk spurt

 for first packet in talk spurt, playout time is:

playout-timei = ti + di + Kvi
remaining packets in talkspurt are played out
periodically

Multmedia Networking 5-34

Adaptive playout delay (3)
Q: How does receiver determine whether packet
is first in a talkspurt?
 if no loss, receiver looks at successive
timestamps
 difference of successive stamps > 20 msec -->talk
spurt begins.
 with loss possible, receiver must look at both
time stamps and sequence numbers
 difference of successive stamps > 20 msec and
sequence numbers without gaps --> talk spurt
begins.

Multmedia Networking 7-35

 VoiP: recovery from packet loss (1)
Challenge:
recover from packet loss given small tolerable
delay between original transmission and playout
 each ACK/NAK takes ~ one RTT
 alternative: Forward Error Correction (FEC)
 send enough bits to allow recovery without
retransmission (recall two-dimensional parity in Ch.
5)

simple FEC
 for every group of n chunks, create redundant chunk by
exclusive OR-ing n original chunks
 send n+1 chunks, increasing bandwidth by factor 1/n
 can reconstruct original n chunks if at most one lost
chunk from n+1 chunks, with playout delay

Multmedia Networking 7-36

VoiP: recovery from packet loss (2)
another FEC scheme:
 “piggyback lower
quality stream”
 send lower resolution
audio stream as
redundant information
 e.g., nominal
stream PCM at 64 kbps
and redundant stream
GSM at 13 kbps
 non-consecutive loss: receiver can conceal loss
 generalization: can also append (n-1)st and (n-2)nd low-bit
rate chunk
Multmedia Networking 7-37
 VoiP: recovery from packet loss (3)

interleaving to conceal
loss:  if packet lost, still have
 audio chunks divided into most of every original
smaller units, e.g. four 5 chunk
msec units per 20 msec  no redundancy overhead,
audio chunk but increases playout
 packet contains small units delay
from different chunks Multmedia Networking 7-38
Voice-over-IP: Skype
 proprietary application- Skype clients (SC)
layer protocol (inferred
via reverse
engineering)
 encrypted msgs
 P2P components: Skype
login server supernode (SN)
 clients: skype peers
connect directly to supernode
overlay
each other for VoIP network
call
 super nodes (SN):
skype peers with
special functions
 overlay network: among
SNs to locate SCs
 login server
Application Layer 2-39
P2P voice-over-IP: skype
skype client operation:
1. joins skype network by
contacting SN (IP
address cached) using
Skype
TCP login server

2. logs-in (usename,
password) to
centralized skype login
server
3. obtains IP address for
callee from SN, SN
overlay
 or client buddy list
4. initiate call directly to
callee
Application Layer 2-40
Skype: peers as relays
 problem: both Alice, Bob
are behind “NATs”
 NAT prevents outside peer
from initiating connection
to insider peer
 inside peer can initiate
connection to outside

 relay solution: Alice, Bob

maintain open connection
to their SNs
 Alice signals her SN to
connect to Bob
 Alice’s SN connects to Bob’s
SN
 Bob’s SN connects to Bob
over open connection Bob
initially initiated to his SN
Application Layer 2-41
Multimedia networking: outline
7.1 multimedia networking applications
7.2 streaming stored video
7.3 voice-over-IP
7.4 protocols for real-time conversational
applications: RTP, SIP
7.5 network support for multimedia

Multmedia Networking 7-42

Real-Time Protocol (RTP)

 RTP specifies  RTP runs in end

packet structure for systems
packets carrying  RTP packets
audio, video data encapsulated in UDP
 RFC 3550 segments
 RTP packet  interoperability: if two
provides VoIP applications run
 payload type RTP, they may be
identification able to work together
 packet sequence
numbering
 time stamping

Multmedia Networking 7-43

RTP runs on top of UDP
RTP libraries provide transport-layer interface
that extends UDP:
• port numbers, IP addresses
• payload type identification
• packet sequence numbering
• time-stamping

Multmedia Networking 5-44

RTP example
example: sending 64  RTP header
kbps PCM-encoded indicates type of
voice over RTP audio encoding in
application collects each packet
encoded data in chunks,  sender can change
e.g., every 20 msec = encoding during
160 bytes in a chunk conference
audio chunk + RTP
 RTP header also
header form RTP contains sequence
packet, which is numbers,
encapsulated in UDP timestamps
segment

Multmedia Networking 7-45

RTP and QoS
 RTP does not provide any mechanism to
ensure timely data delivery or other QoS
guarantees
 RTP encapsulation only seen at end systems
(not by intermediate routers)
 routers provide best-effort service, making
no special effort to ensure that RTP
packets arrive at destination in timely
matter

Multmedia Networking 7-46

 RTP header
payload sequence Synchronization Miscellaneous
time stamp
type number Source ID fields
type
payload type (7 bits): indicates type of encoding currently
being
used. If sender changes encoding during call, sender
informs receiver via payload type field
Payload type 0: PCM mu-law, 64 kbps
Payload type 3: GSM, 13 kbps
Payload type 7: LPC, 2.4 kbps
Payload type 26: Motion JPEG
Payload type 31: H.261
Payload type 33: MPEG2 video

sequence # (16 bits): increment by one for each RTP packet

sent
detect packet loss, restore packet sequence
Multmedia Networking 5-47
RTP header
payload sequence Synchronization Miscellaneous
time stamp
type number Source ID fields
type

 timestamp field (32 bits long): sampling instant

of first byte in this RTP data packet
 for audio, timestamp clock increments by one for
each sampling period (e.g., each 125 usecs for 8
KHz sampling clock)
 if application generates chunks of 160 encoded
samples, timestamp increases by 160 for each RTP
packet when source is active. Timestamp clock
continues to increase at constant rate when source
is inactive.

 SSRC field (32 bits long): identifies source of RTP

stream. Each stream in RTP session has distinct SSRC

Multmedia Networking 7-48