Joint Probability

Distributions
Outlines
 Two Discrete/Continuous Random Variables
 Joint Probability Distributions

 Marginal Probability Distributions

 Conditional Probability Distributions

 Independence

 Multiple Discrete/Continuous Random Variables

 Joint Probability Distributions

 Multinomial Probability Distribution

 Covariance and Correlation

 Bivariate Normal Distribution
 Linear Combination of random variables
Joint Probability
Distributions
 In general, if X and Y are two random variables,
the probability distribution that defines their
simultaneous behavior is called a joint
probability distribution.
 For example: X : the length of one dimension of
an injection-molded part, and Y : the length of
another dimension. We might be interested in
 P(2.95  X  3.05 and 7.60  Y  7.80).
Two Discrete Random
Variables
 Joint Probability Distributions
 Marginal Probability Distributions
 Conditional Probability Distributions
 Independence
Joint Probability
Distributions
 The joint probability distribution of two
random variables =bivariate probability
distribution.
 The joint probability distribution of two
discrete random variables is usually
written as P(X=x, Y=y).
Marginal Probability
Distributions
 Marginal Probability Distribution: the
individual probability distribution of a
random variable.
Marginal Probability
Distributions
 Example: The marginal probability
distribution for X and Y.
y=num x=number of bars of signal strength
ber of 1 2 3 Marginal
times probabili
city ty
name is distributi
stated on of Y

4 0.15 0.1 0.05 0.3

3 0.02 0.1 0.05 0.17
2 0.02 0.03 0.2 0.25
P(X=3)
1 0.01 0.02 0.25 0.28
0.2 0.25 0.55
Conditional Probability
Distributions
 When two random variables are defined
in a random experiment, knowledge of
one can change the probabilities of the
other.
Conditional Mean and
Variance
Conditional Mean and
Variance
 Example: From the previous example,
calculate
P(Y=1|X=3), E(Y|1), and V(Y|1).
P (Y 1 | X 3) P ( X 3, Y 1) / P ( X 3)
 f x , y (3,1) / f x (3) 0.25 / 0.55 0.454

E (Y | 1)  yfY |1 ( y )
y

1(0.05)  2(0.1)  3(0.1)  4(0.75) 3.55

V (Y | 1)  ( y  Y | x ) 2 fY |1 ( y )
y

(1  3.55) 2 0.05  (2  3.55) 2 0.1  (3  3.55) 2 0.1  (4  3.55) 2 0.75

0.748
Independence
 In some random experiments, knowledge of the
values of X does not change any of the
probabilities associated with the values for Y.
 If two random variables are independent, then
Multiple Discrete Random
Variables
 Joint Probability Distributions
 Multinomial Probability Distribution
Joint Probability
Distributions
 In some cases, more than two random
variables are defined in a random
experiment.

 Marginal probability mass function

Joint Probability
Distributions
 Mean and Variance
Joint Probability
Distributions
 Conditional Probability Distributions

 Independence
Multinomial Probability
Distribution
 A joint probability distribution for multiple
discrete random variables that is quite useful in
an extension of the binomial.
Multinomial Probability
Distribution
 Example: Of the 20 bits received, what is the probability
that 14 are Excellent, 3 are Good, 2 are Fair, and 1 is Poor?
Assume that the classifications of individual bits are
independent events and that the probabilities of E, G, F,
and P are 0.6, 0.3, 0.08, and 0.02, respectively.

 One sequence of 20 bits that produces the specified

numbers of bits in each 0class
.6140.33can
0.082 0be
.021represented
2.708 10  9 as:
EEEEEEEEEEEEEEGGGFFP 20!
2325600
 P(EEEEEEEEEEEEEEGGGFFP)= 14!3!2!1!
9
 P (14 E ' s,3G ' s,2 F ' s,1P ) 23256002.708 10 0.0063
 The number of sequences (Permutation of similar objects)=
Two Continuous Random
Variables
 Joint Probability Distributions
 Marginal Probability Distributions
 Conditional Probability Distributions
 Independence
Joint Probability
Distributions
Joint Probability
Distributions
 Example: X: the time until a computer server connects to
your machine , Y: the time until the server authorizes you
as a valid user. Each of these random variables measures
the wait from a common starting time and X <Y. Assume
that the
f XY ( xjoint
, y ) 6probability density
10 6 exp( 0.001x  0.002 yfunction
), x  y for X and Y is

 The probability that X<1000 and Y<2000 is:

Marginal Probability
Distributions
 Similar to joint discrete random
variables, we can find the marginal
probability distributions of X and Y from
the joint probability distribution.
Marginal Probability
Distributions
 Example: For the random variables in the previous
example, calculate the probability that Y exceeds 2000
milliseconds.
Conditional Probability
Distributions
Conditional Probability
Distributions
 Example: For the random variables in the previous example,
( fY | x ( y ))
determine the conditional probability density function for Y given
that X=x f XY ( x, y )
fY | x ( y )  , for f X ( x)  0
f X ( x)

 Determine P(Y>2000|x=1500)
Conditional Probability
Distributions
 Mean and Variance
Conditional Probability
Distributions
 Example: For the random variables in the previous
example, determine the conditional mean for Y given that
x=1500
Independence
Independence
 Example: Let the random variables X and Y denote the lengths of
two dimensions of a machined part, respectively.
 Assume that X and Y are independent random variables, and the
distribution of X is normal with mean 10.5 mm and variance
0.0025 (mm)2 and that the distribution of Y is normal with mean
3.2 mm and variance 0.0036 (mm)2.
 Determine the probability that 10.4 < X < 10.6 and 3.15 < Y <
3.25.
 Because X,Y are independent
Multiple Continuous Random
Variables
Multiple Continuous Random
Variables
 Marginal Probability
Multiple Continuous Random
Variables
 Mean and Variance

 Independence
Covariance and Correlation
 When two or more random variables are
defined on a probability space, it is
useful to describe how they vary
together.
 It is useful to measure the relationship
between the variables.
Covariance
 Covariance is a measure of linear relationship
between the random variables.

 The expected value of a function of two random

variables
h(X, Y ).
Covariance
 
E[(Y  Y )( X   X )]  ( x   X )( y  Y ) f XY ( x, y )dxdy
  
 
 [ xy   X y  xY   X Y ] f XY ( x, y )dxdy (1)
  

Now
 
 

  
 X yf XY ( x , y ) dxdy   X  
   
yf XY ( x , y ) dxdy

(2)

 
From E (h( y ))  h( y ) f XY ( x, y )dxdy
  
 
For h( y )  y; E ( y )  yf XY ( x, y )dxdy Y
  
   
Substitute in (2), 
  
X yf XY ( x, y )dxdy  X Y , and x
  
y f XY ( x, y )dxdy  X Y

 
Substitute in (1), E[(Y  Y )( X   X )]  xyf XY ( x, y )dxdy   X Y   X Y   X Y
  
 
 xyf XY ( x, y )dxdy   X Y E ( XY )   X Y
  
Covariance
Covariance
 Example: For the discrete random variables X, Y with the
 XY and
joint distribution shown in 
Fig.
XY Determine
Correlation
 The correlation is a measure of the linear
relationship between random variables.
 Easier to interpret than the covariance.
Correlation
 For independent random variables
Correlation
1
 Example: Two random variables
f XY ( x , y )  , calculate
xy
16
the covariance and correlation between X and Y.
Bivariate Normal
Distribution

Correlation
Bivariate Normal
Distribution
 Marginal distributions

 Dependence
Bivariate Normal
Distribution
 Conditional probability
Y Y
 Y | x  Y   X   x
X X
 Y2| x  Y2 (1   2 )
Bivariate Normal
Distribution
Ex. Suppose that the X and Y dimensions of an injection-modeled
 xnormal
part have a bivariate 0.04,  ydistribution
0.08,  x 3.00,  y 7.70,  0.8
with
Find the P(2.95<X<3.05,7.60<Y<7.80)
Bivariate Normal
Distribution
 Ex. Let X, Y : milliliters of acid and base needed for
equivalence, respectively. Assume X and Y have a bivariate
normal distribution with
 x 5,  y 2,  x 120,  y 100,  0.6

 Covariance between X and Y

 Marginal probability distribution of X
 P(X<116)
 P(X|Y=102)
 P(X<116|Y=102)
Linear Combination of random
variables
Linear Combination of random
variables
 Mean and Variance
Linear Combination of random
variables
Ex. A semiconductor product consists of 3 layers. The
variances in thickness of the first, second, and third layers
are 25,40,30 nm2 . What is the variance of the thickness of
the final product?

Let X1, X2, X3, and X be random variables that denote the
thickness of the respective layers, and the final product.

V(X)=V(X1)+V(X2)+V(X3)=25+40+30=95 nm2
Homework
1. The time between surface finish problems in a galvanizing
process is exponentially distributed with a mean of 40 hours. A
single plant operates three galvanizing lines that are assumed to
operate independently.
a) What is the probability that none of the lines experience a surface finish
problem in 40 hours of operation?
b) What is the probability that all three lines experience two surface finish
problems between 20 and 40 hours after starting the operation?
2. Suppose X and Y have a bivariate normal distribution with
 x 0.04,  y 0.08,  x 3.00,  y 7.70,  0.
Determine the
following:
a) P(2.95<X<3.05)
b) P(7.60<Y<7.80)
c) P(2.95<X<3.05,7.60<Y<7.80)