Department of Computer Engineering, Faculty of Engineering,

Kasetsart University, THAILAND

1st Semester 2014

(Aug – Dec)
Assoc. Prof. Anan Phonphoem, Ph.D.
http://www.cpe.ku.ac.th/~anan

Department of Computer Engineering, Faculty of Engineering,

Kasetsart University, THAILAND
Definition:
FX(x) = P[X  x]
• Contain complete information about
the probability model of the random variable
• PMF CDF

Anan Phonphoem, Dept.of Computer Engineering, Kasetsart Universty 4

 FFXX(x
(x21)) == P[X
P[X xx21]] FX() = P[X ]
PX(x)

- x1 x2 x 

Anan Phonphoem, Dept.of Computer Engineering, Kasetsart Universty 5

Theorem: For a discrete random variable X
with SX={x1,x2,…} & x1  x2  …

1) Fx(-) = 0 and Fx() = 1

2) x’  x, Fx(x’)  Fx(x)
3) For xi Sx and  = +small number
Fx(xi) – Fx(xi - ) = Px(xi)
4) Fx(x) = Fx(xi) x, xi  x < xi+1

Anan Phonphoem, Dept.of Computer Engineering, Kasetsart Universty 6

 FX (-) PX(x) FX ()

- 
x

Anan Phonphoem, Dept.of Computer Engineering, Kasetsart Universty 7

PX(x) FX (x) FX (x’)

- 
x
x x’
Monotonic Increasing
Anan Phonphoem, Dept.of Computer Engineering, Kasetsart Universty 8
PX(x) FX (xi)
FX (xi - )

- 
 xi x

Discontinuity = PX(x)
Anan Phonphoem, Dept.of Computer Engineering, Kasetsart Universty 9
PX(x) FX (xi)

- 
x
xi xi+1
Horizon line
Anan Phonphoem, Dept.of Computer Engineering, Kasetsart Universty 10
Theorem: For a discrete random variable X
with SX={x1,x2,…} & x1  x2  …

1) Fx(-) = 0 and Fx() = 1  From 0 to 1

2) x’  x, Fx(x’)  Fx(x)  Monotonic Increasing
3) For xi Sx and  = +small number
Fx(xi) – Fx(xi - ) = Px(xi)  Discontinuity = PX(x)
4) Fx(x) = Fx(xi) x, xi  x < xi+1  Horizon line

Anan Phonphoem, Dept.of Computer Engineering, Kasetsart Universty 11

• For a Binomial RV, # of fail programs in 5
tests with p = 0.2

PK(k) = () 5
k
(0.2)k(0.8)5-k k = 0,1,2,…,5

0 Otherwise

3 July 2013 Anan Phonphoem, Dept.of Computer Engineering, Kasetsart Universty 12

 k PK(k) k PK(k)
0 0.33 3 0.05
1 0.41 4 0.01
2 0.20 5 0
. . .
PK(k) 0.5 FK(k) 1
.
0.25

0
0.5

0
.
0 1 2 3 4 5 0 1 2 3 4 5
k k
Note: FK(k) is continuous from right
3 July 2013 Anan Phonphoem, Dept.of Computer Engineering, Kasetsart Universty 13
 b  a,
FX(b) - FX(a) = P[a < X  b]
Difference of the CDF is the probability that RV takes on the
value between two points
PX(x)

x
a b
3 July 2013 Anan Phonphoem, Dept.of Computer Engineering, Kasetsart Universty 14
• Study RV  Average
• What is the average of an RV?
• A single number that describes the RV
• An example of statistic
• What is Statistic?
• Numbers that collect all information of things
under our interesting
• Averages: mean, mode, and median

3 July 2013 Anan Phonphoem, Dept.of Computer Engineering, Kasetsart Universty 16

 Picture from http://rchsbowman.wordpress.com/2008/09/15/statistics-notes-%E2%80%94-measures-of-variation/

Can you identify the Mean, Mode, and Median for each team?

3 July 2013 Anan Phonphoem, Dept.of Computer Engineering, Kasetsart Universty 17

• Mean:
• Sum / #terms

• Mode:
• Most common value
• PX(xmod)  PX(x) x

• Median:
• The middle of the data set
• P[X < xmed] = P[X > xmed]

3 July 2013 Anan Phonphoem, Dept.of Computer Engineering, Kasetsart Universty 18

 • Adding all measurements / #terms
Definition:
E[X] = X =  xPX(x)
xSX

3 July 2013 Anan Phonphoem, Dept.of Computer Engineering, Kasetsart Universty 19

• Experiment  Random Variable X
• Perform n independent trials
• The value X takes on ith trial  x(i)
The average
x(3)
x(2) x(1) n
mn = 1  x(i)
x(5) n i=1
x(4)
x(n)

3 July 2013 Anan Phonphoem, Dept.of Computer Engineering, Kasetsart Universty 20

• Each x(i) takes value in the set Sx
• Out of n trials, assume xS occurs N X X times

X(x(1)) = x
x(3)
x(2) x(1)

x(4) x(5) -2.0 -1.5 -1.0 -0.5 0 0.5

x
x(n)
SX

3 July 2013 Anan Phonphoem, Dept.of Computer Engineering, Kasetsart Universty 21

 n
mn = 1  x(i)
n i=1

1   NX
mn = n xSxNxx = xSxn x
NA NX
P[A] = lim PX(x) = lim
n→ n n→ n

limm n =  xPX(x)
n→ xS x

E[X] =  xPX(x)
xSX

3 July 2013 Anan Phonphoem, Dept.of Computer Engineering, Kasetsart Universty 22

 E[X] =  xPX(x)
xSX

• Example:
1/4 t=0
PT(t) = 3/4 t=2
0 Otherwise
• E[T] = ?
= 0(1/4) + 2(3/4) = 3/2

3 July 2013 Anan Phonphoem, Dept.of Computer Engineering, Kasetsart Universty 23

 PX(x)
PT(t) Probability MASS Function
1

0.5

0 t
0 1 2

E[X]
3 July 2013 Anan Phonphoem, Dept.of Computer Engineering, Kasetsart Universty 24
 1–p x=0
PX(x) = p x=1
0 Otherwise

E[X] =  xPX(x)
xSX

Sx = {0,1}
E[X] = 0(1-p) + 1(p)
=p

3 July 2013 Anan Phonphoem, Dept.of Computer Engineering, Kasetsart Universty 25

 p(1 – p)x-1 x = 1,2,3,…
PX(x) = 0 Otherwise

E[X] =  xPX(x)
xSX
 
=  xp(1-p)x-1 =  xpqx-1
x=1 x=1

= p

xqx-1
? 1
E[X] = p
x=1

3 July 2013 Anan Phonphoem, Dept.of Computer Engineering, Kasetsart Universty 26

 n
Sn =  xqx-1
x=1
Sn = 1 + 2q + 3q2 + 4q3 + … + nqn-1
q Sn= q + 2q2 + 3q3 + … + (n-1)qn-1 + nqn
(1-q) Sn= 1 + q + q2 + q3 + … + qn-1 - nqn
q(1-q) Sn= q + q 2 + q3 + … +qn – nqn+1
(1-q)2 Sn= 1 – (n+1)qn + nqn+1
n
0 n+1 0
(n+1)q nq
Sn = 1 2 - 2
+
(1-q) (1-q) (1-q)2
1 
Sn = E[X]= p xqx-1 = 1
p2 x=1 p
3 July 2013 Anan Phonphoem, Dept.of Computer Engineering, Kasetsart Universty 27
From example:
• Find the number of tests until find a fail
program
• We have p = 0.2  2/10  1/5
• E[X] = 1/p = 5
• Intuitively, on average, we will find the fail
program after 5 tests.

3 July 2013 Anan Phonphoem, Dept.of Computer Engineering, Kasetsart Universty 28

 xe- x = 0,1,2…
PX(x) = x!
0 Otherwise

 
E[X] =  xPX(x) = x x -
e
x!
e
x=0 x=0
 
= 
 e- x-1
= 
k
e-
(x-1)! k!
(x=0)  0 x=1 k=0

=  e e- E[X] = 
3 July 2013 Anan Phonphoem, Dept.of Computer Engineering, Kasetsart Universty 29
 Uniform
1/(j-k+1) x = k,k+1,k+2,..,j
Equiprobable E[X] =(j+k)
outcomes 2
0 Otherwise

Bernoulli 1–p x=0

p x=1 E[X] = p
Pass/Fail 0 Otherwise

Geometric
p(1 – p)x-1 x = 1,2,3,…
E[X] = 1/p
# tests until fail 0 Otherwise
3 July 2013 Anan Phonphoem, Dept.of Computer Engineering, Kasetsart Universty 30
 Binomial n
( ) p (1 – p)
x
x n-x x=1,2,…,n
E[X] = np
# fails in n tests 0 Otherwise

Pascal x-1
( ) p (1 – p) k x-k x = k,k+1,…
k-1
# tests until k E[X] = k/p
fails 0 Otherwise
Poisson (T)xe-(T) x = 0,1,2…
E[X] = 
occurrence in a x!  = T
period 0 Otherwise

3 July 2013 Anan Phonphoem, Dept.of Computer Engineering, Kasetsart Universty 31

 Theorem: Let p = /n ( > 0 and n > )
Binomial PMF  Poisson PMF (parameter )

PX(x) =
( ) p (1 – p)
x
x n-x x=0,1,2,…,n

0 Otherwise

3 July 2013 Anan Phonphoem, Dept.of Computer Engineering, Kasetsart Universty 32

 p = /n
n  x 
PX(x) =
() x
( n
) (1 – n
)n-x x=0,1,2,…,n

0 Otherwise
n(n-1)…(n-x+1) x (1 –  )n-x
= n
e-
nx x!
(1 –  )n
n
lim (n-j) = 1
n→ n (1 –  )x
n 1

xe- x = 0,1,2…
lim PX(x) = x!
n→
0 Otherwise
3 July 2013 Anan Phonphoem, Dept.of Computer Engineering, Kasetsart Universty 33
 s1 s2 s3 X(s)
s4 s5 Y=g(X)
sn

-2.0 -1.5 -1.0 -0.5

x0
SX
0 1 2 3 4
y5 6 7 8

SY
3 July 2013 Anan Phonphoem, Dept.of Computer Engineering, Kasetsart Universty 35
• From sample values of the random variable,
use these values to compute other quantities.
• Y = g(X)
• Example:
• Find a decibel value form signal-to-noise ratio
• Watt  dB
Calculate 0.1W relative to 1 mW (milliwatt)
100 mW
 GdB = 10 log10( ) = 20 dB 20 dBm
1 mW

3 July 2013 Anan Phonphoem, Dept.of Computer Engineering, Kasetsart Universty 36

• Random Variable X = # pages in one fax
• PX(x) = number of pages in each fax
• Charging plan
• 1st page = 10 Baht
• 2nd page = 9 Baht
• …
• 5th page = 6 Baht
• 6 – 10 pages = 50 Baht
• Find the charge in Baht for sending one fax
3 July 2013 Anan Phonphoem, Dept.of Computer Engineering, Kasetsart Universty 37
• Random Variable Y = the charge in Baht
for sending one fax

10.5X – 0.5X2 1X 5

Y = g(X) = 50 6  X  10

3 July 2013 Anan Phonphoem, Dept.of Computer Engineering, Kasetsart Universty 38

 Theorem:
PY(y) =  PX(x)
x:g(x)=y

P[Y=y] =  of all outcomes X = x for which Y = y

3 July 2013 Anan Phonphoem, Dept.of Computer Engineering, Kasetsart Universty 39

• Suppose all faxes contain 1,2,3, or 4 pages
with equal probability
• Find PMF and E[Y]

¼ x = 1,2,3,4
PX(x) = 0 Otherwise

• SX = {1,2,3,4}
• SY = {10,19,27,34}

3 July 2013 Anan Phonphoem, Dept.of Computer Engineering, Kasetsart Universty 40

 ¼ X=1 Y=10

¼ X=2 Y=19
¼ X=3 Y=27
¼
X=4 Y=34

¼ y = 10,19,27,34
PY(y) = 0 Otherwise

E[Y] = (1/4)(10 + 19 + 27 + 34) = 22.5 Baht

3 July 2013 Anan Phonphoem, Dept.of Computer Engineering, Kasetsart Universty 41

 0.15 x = 1,2,3,4
PX(x) = 0.1 x = 5,6,7,8
0 Otherwise

• Find PMF and E[Y]

0.2
PX(x)

0.1

0
0 2 4 6 8
x
3 July 2013 Anan Phonphoem, Dept.of Computer Engineering, Kasetsart Universty 42
 X=1 Y=10
0.15

0.15 X=2 Y=19

0.15 X=3 Y=27
0.15
X=4 Y=34
0.1
X=5 Y=40
0.1
0.1 X=6
0.1
X=7 Y=50

X=8
3 July 2013 Anan Phonphoem, Dept.of Computer Engineering, Kasetsart Universty 43
 From Theorem
0.15 y = 10,19,27,34
PY(5) = 0.1 y = 40
PY(y) = PY(6)+PY(7)+PY(8) = 0.3 y = 50
0 Otherwise

0.4
PY(y)

E[Y] = 0.15(10+19+27+34)
0.2
+ 0.1(40) + 0.3(50)
0 = 32.5
10 19 27 34 40 50
y
3 July 2013 Anan Phonphoem, Dept.of Computer Engineering, Kasetsart Universty 44