NPTEL- Probability and Distributions

MODULE 6
RANDOM VECTOR AND ITS JOINT DISTRIBUTION
LECTURE 30
Topics
6.5 EXPECTATIONS AND MOMENTS
6.6 JOINT MOMENT GENERATING FUNCTION

Theorem 5.5

Under the above notations

(i) 𝐸 𝐸 𝜓 𝑌 𝑍 =𝐸 𝜓 𝑌 ;
(ii) Var 𝐸 𝜓 𝑌 𝑍 + 𝐸 Var 𝜓 𝑌 𝑍 = Var 𝜓 𝑌 .

Proof. We will provide the proof for the absolutely continuous case. The proof for the
discrete case follows in the similar fashion.

(i) Note that

𝐸 𝐸 𝜓 𝑌 𝑍 = 𝐸 𝜓∗ 𝑍 ,

where 𝜓 ∗ ∙ is defined by (5.2) and (5.3). Therefore

𝐸 𝐸 𝜓 𝑌 𝑍 = 𝜓 ∗ 𝑧 𝑓𝑍 𝑧 𝑑𝑧
ℝ𝑝 2

= 𝜓 𝑦 𝑓𝑌 𝑍 𝑦 𝑧 𝑑𝑦 𝑓𝑍 𝑧 𝑑𝑧
ℝ𝑝 2 ℝ𝑝 1

= 𝜓 𝑦 𝑓𝑌,𝑍 𝑦, 𝑧 𝑑𝑦𝑑𝑧
ℝ𝑝 2 ℝ 𝑝 1

= 𝐸(𝜓 𝑌 ).

Dept. of Mathematics and Statistics Indian Institute of Technology, Kanpur 1

NPTEL- Probability and Distributions

(ii) Let 𝜓 ∗ 𝑍 = 𝐸 𝜓 𝑌 𝑍 . Then, by (i),

2
Var 𝜓 𝑌 =𝐸 𝜓 𝑌 − 𝐸(𝜓 𝑌 )

2
=𝐸 𝐸 𝜓 𝑌 −𝐸 𝜓 𝑌 𝑍 (5.4)

2
2
𝐸 𝜓 𝑌 − 𝐸(𝜓 𝑌 ) 𝑍 =𝐸 𝜓 𝑌 −𝐸 𝜓 𝑌 𝑍 +𝐸 𝜓 𝑌 𝑍 −𝐸 𝜓 𝑌 |𝑍

2 2
=𝐸 𝜓 𝑌 − 𝐸 𝜓 𝑌 |𝑍 |𝑍 + 𝐸 𝜓 𝑌 𝑍 − 𝐸 𝜓 𝑌

+2 𝐸 𝜓 𝑌 𝑍 − 𝐸 𝜓 𝑌 𝐸 𝜓 𝑌 −𝐸 𝜓 𝑌 𝑍 𝑍

2
= Var 𝜓 𝑌 𝑍 + 𝐸 𝜓 𝑌 𝑍 − 𝐸 𝐸 𝜓 𝑌 𝑍 . (5.5)

Combining (5.4) and (5.5), we get

2
Var 𝜓 𝑌 = 𝐸 Var 𝜓 𝑌 𝑍 +𝐸 𝐸 𝜓 𝑌 𝑍 −𝐸 𝐸 𝜓 𝑌 𝑍

= 𝐸 Var 𝜓 𝑌 𝑍 + Var 𝐸 𝜓 𝑌 𝑍 .▄

Remark 5.1

If 𝑌 and 𝑍 are independent then

𝐸 𝜓 𝑌 𝑍 =𝐸 𝜓 𝑌 and Var 𝜓 𝑌 𝑍 = Var 𝜓 𝑌 .

Example 5.1

Let 𝑋 = 𝑋1 , 𝑋2 , 𝑋3 be a discrete type random vector with p.m.f.

𝑥1 𝑥2 𝑥3
, if 𝑥1 , 𝑥2 , 𝑥3 ∈ 1, 2 × 1, 2, 3 × {1, 3}
𝑓𝑋 𝑥1 , 𝑥2 , 𝑥3 = 72 .
0, otherwise

(i) Let 𝑌1 = 2𝑋1 − 𝑋2 + 3𝑋3 and 𝑌2 = 𝑋1 − 2𝑋2 + 𝑋3 . Find the correlation coefficient
between 𝑌1 and 𝑌2 ;
(ii) For a fixed 𝑥2 ∈ 1, 2, 3 , find 𝐸 𝑌 𝑋2 = 𝑥2 and Var 𝑌 𝑋2 = 𝑥2 , where 𝑌 = 𝑋1 𝑋3 .

Dept. of Mathematics and Statistics Indian Institute of Technology, Kanpur 2

NPTEL- Probability and Distributions

Solution.

(i) From Example 4.1 (i) we know that 𝑋1 , 𝑋2 and 𝑋3 are independent.
Therefore Cov(𝑋1 , 𝑋2 ) = Cov(𝑋1 , 𝑋3 ) = Cov(𝑋2 , 𝑋3 ) = 0 . Also Cov(𝑋𝑖 , 𝑋𝑖 ) =
Var 𝑋𝑖 , 𝑖 = 1, 2, 3. Using Theorem 5.2 (ii) we have

Cov(𝑌1 , 𝑌2 ) = 2 Var 𝑋1 − 5 Cov(𝑋1 , 𝑋2 ) + 2 Var 𝑋2 + 5 Cov(𝑋1 , 𝑋3 )

+3Var 𝑋3 − 7Cov(𝑋2 , 𝑋3 )

= 2Var 𝑋1 + 2Var 𝑋2 + 3Var 𝑋3 .

From the solution of Example 4.1 (ii) we have

𝑥1 𝑥2
, if 𝑥1 ∈ {1, 2} , if 𝑥2 ∈ {1, 2, 3}
𝑓𝑋1 𝑥1 = 3 , 𝑓𝑋2 𝑥2 = 6
0, otherwise 0, otherwise

and
𝑥3
, if 𝑥3 ∈ {1, 3}
𝑓𝑋3 𝑥3 = 4 .
0 , otherwise

Therefore

𝑥12 (12 + 22 ) 5
𝐸 𝑋1 = 𝑥1 𝑓𝑋1 𝑥1 = = =
3 3 3
𝑥 1 ∈𝑆𝑋 1 𝑥 1 ∈{1,2}

𝑥13 (13 + 23 )
𝐸 𝑋12 = 𝑥12 𝑓𝑋1 𝑥1 = = =3
3 3
𝑥 1 ∈𝑆𝑋 1 𝑥 1 ∈{1,2}

𝑥22 (12 + 22 + 32 ) 7
𝐸 𝑋2 = 𝑥2 𝑓𝑋2 𝑥2 = = =
6 6 3
𝑥 2 ∈𝑆𝑋 2 𝑥 2 ∈{1,2,3}

𝑥23 (13 + 23 + 33 )
𝐸 𝑋22 = 𝑥22 𝑓𝑋2 𝑥2 = = =6
6 6
𝑥 2 ∈𝑆𝑋 2 𝑥 2 ∈{1,2,3}

𝑥32 (12 + 32 ) 5
𝐸 𝑋3 = 𝑥3 𝑓𝑋3 𝑥3 = = =
4 4 2
𝑥 3 ∈𝑆𝑋 3 𝑥 3 ∈{1,3}

𝑥33 (13 + 33 )
𝐸 𝑋32 = 𝑥32 𝑓𝑋3 𝑥3 = = =7
4 4
𝑥 3 ∈𝑆𝑋 3 𝑥 3 ∈{1,3}

Dept. of Mathematics and Statistics Indian Institute of Technology, Kanpur 3

NPTEL- Probability and Distributions

2
Var 𝑋1 = 𝐸 𝑋12 − (𝐸 𝑋1 )2 =
9
5
Var 𝑋2 = 𝐸 𝑋22 − (𝐸 𝑋2 )2 =
9
and

2 3
Var 𝑋3 = 𝐸 𝑋32 − 𝐸 𝑋3 = .
4
Therefore,

4 10 9 137
Cov(𝑌1 , 𝑌2 ) = + + = .
9 9 4 36
Also, by Corollary 5.1,

Var 𝑌1 = Var(2𝑋1 − 𝑋2 + 3𝑋3 )

= 4 Var 𝑋1 + Var 𝑋2 + 9 Var 𝑋3

8 5 27
= + +
9 9 4
295
=
36
and

Var 𝑌2 = Var(𝑋1 − 2𝑋2 + 𝑋3 )

= Var 𝑋1 + 4Var 𝑋2 + Var 𝑋3

2 20 3
= + +
9 9 4
115
= .
36
Therefore

Cov 𝑌1 , 𝑌2
𝜌 𝑌1 , 𝑌2 =
Var 𝑌1 Var(𝑌2 )

137
=
295 115

Dept. of Mathematics and Statistics Indian Institute of Technology, Kanpur 4

NPTEL- Probability and Distributions

= 0.7438 ⋯

(ii) Since 𝑋1 , 𝑋2 and 𝑋3 are independent it follows that (𝑋1 , 𝑋3 ) and 𝑋2 are
independent. This in turn implies that 𝑌 = 𝑋1 , 𝑋3 and 𝑋2 are independent.
Therefore 𝐸(𝑌 𝑋2 = 𝑥2 ) = 𝐸 𝑌 and Var(𝑌 𝑋2 = 𝑥2 ) = Var 𝑌 . Now

𝐸 𝑌 = 𝐸 𝑋1 𝑋3

= 𝐸(𝑋1 )𝐸(𝑋3 ) (using Theorem (5.3))

25
= .
6
Var 𝑌 = Var 𝑋1 𝑋3

= Var(𝐸 𝑋1 𝑋3 𝑋3 + 𝐸(Var 𝑋1 𝑋3 𝑋3 )

= Var(𝑋3 𝐸 𝑋1 𝑋3 ) + 𝐸 𝑋32 Var(𝑋1 𝑋3 )

= Var 𝑋3 𝐸 𝑋1 + 𝐸 𝑋32 Var 𝑋1 (Remark 5.1)

5 2
= Var 𝑋3 + 𝐸 𝑋32
3 9
25 2
= Var 𝑋3 + 𝐸(𝑋32 )
9 9
75 14
= +
36 9
131
= .▄
36
Example 5.2

Let 𝑋 = 𝑋1 , 𝑋2 , 𝑋3 be an absolutely continuous type random vector with p.d.f.

1
, if 0 < 𝑥3 < 𝑥2 < 𝑥1 < 1
𝑓𝑋 𝑥1 , 𝑥2 , 𝑥3 = 𝑥1 𝑥2 .
0, otherwise

(i) Let 𝑌1 = 2𝑋1 − 𝑋2 + 3𝑋3 and 𝑌2 = 𝑋1 − 2𝑋2 + 𝑋3 . Find 𝜌 𝑌1 , 𝑌2 ;

(ii) For a fixed 𝑥1 ∈ 0, 1 find 𝐸(𝑌 𝑋1 = 𝑥1 ) and Var(𝑌 𝑋1 = 𝑥1 ), where 𝑌 = 𝑋1 𝑋2 𝑋3 .

Dept. of Mathematics and Statistics Indian Institute of Technology, Kanpur 5

NPTEL- Probability and Distributions

Solution.

(i) As in Example 5.1 (i)

Cov(𝑌1 , 𝑌2 ) = 2 Var 𝑋1 + 2 Var 𝑋2 + 3 Var 𝑋3 − 5 Cov(𝑋1 , 𝑋2 )

+5 Cov(𝑋1 , 𝑋3 ) − 7 Cov(𝑋2 , 𝑋3 ).
1 𝑥1 𝑥2
1 1
𝐸 𝑋1 = 𝑥1 𝑓𝑋 𝑥 𝑑𝑥 = 𝑑𝑥3 𝑑𝑥2 𝑑𝑥1 =
ℝ3 𝑥2 2
0 0 0

1 𝑥1 𝑥2
𝑥1 1
𝐸(𝑋12 ) = 𝑑𝑥3 𝑑𝑥2 𝑑𝑥1 =
𝑥2 3
0 0 0

1 𝑥1 𝑥2
1 1
𝐸 𝑋2 = 𝑑𝑥3 𝑑𝑥2 𝑑𝑥1 =
𝑥1 4
0 0 0

1 𝑥1 𝑥2
𝑥2 1
𝐸(𝑋22 ) = 𝑑𝑥3 𝑑𝑥2 𝑑𝑥1 =
𝑥1 9
0 0 0

1 𝑥1 𝑥2
𝑥3 1
𝐸 𝑋3 = 𝑑𝑥3 𝑑𝑥2 𝑑𝑥1 =
𝑥1 𝑥2 8
0 0 0

1 𝑥1 𝑥2
𝑥32 1
𝐸 𝑋32 = 𝑑𝑥3 𝑑𝑥2 𝑑𝑥1 =
𝑥1 𝑥2 27
0 0 0

1 𝑥1 𝑥2
1
𝐸 𝑋1 𝑋2 = 𝑑𝑥3 𝑑𝑥2 𝑑𝑥1 =
6
0 0 0

1 𝑥1 𝑥2
𝑥3 1
𝐸 𝑋1 𝑋3 = 𝑑𝑥3 𝑑𝑥2 𝑑𝑥1 =
𝑥2 12
0 0 0

1 𝑥1 𝑥2
𝑥3 1
𝐸 𝑋2 𝑋3 = 𝑑𝑥3 𝑑𝑥2 𝑑𝑥1 =
𝑥1 18
0 0 0

1
Var 𝑋1 = 𝐸(𝑋12 ) − (𝐸(𝑋1 ))2 =
12

Dept. of Mathematics and Statistics Indian Institute of Technology, Kanpur 6

NPTEL- Probability and Distributions

7
Var 𝑋2 = 𝐸(𝑋22 ) − (𝐸(𝑋2 ))2 =
144
37
Var 𝑋3 = 𝐸(𝑋32 ) − (𝐸(𝑋3 ))2 =
1728
1
Cov 𝑋1 , 𝑋2 = 𝐸(𝑋1 𝑋2 ) − 𝐸(𝑋1 )𝐸 𝑋2 =
24
1
Cov 𝑋1 , 𝑋3 = 𝐸(𝑋1 𝑋3 ) − 𝐸(𝑋1 )𝐸 𝑋3 =
48
7
Cov 𝑋2 , 𝑋3 = 𝐸(𝑋2 𝑋3 ) − 𝐸(𝑋2 )𝐸 𝑋3 = .
288
Therefore,

1 7 37 5 5 49 31
Cov(𝑌1 , 𝑌2 ) = + + − + − = .
6 72 576 24 48 288 576
Also,

Var 𝑌1 = 4 Var 𝑋1 + Var 𝑋2 + 9 Var 𝑋3 − 4 Cov(𝑋1 , 𝑋2 )

+12 Cov(𝑋1 , 𝑋3 ) − 6 Cov(𝑋2 , 𝑋3 )

1 7 37 1 1 7
= + + − + −
3 144 192 6 4 48
295
= .
576
Var 𝑌2 = Var 𝑋1 + 4Var 𝑋2 + Var 𝑋3 − 4Cov(𝑋1 , 𝑋2 ) + 2Cov(𝑋1 , 𝑋3 ) − 4Cov(𝑋2 , 𝑋3 )

1 7 37 1 1 7
= + + − + −
12 36 1728 6 24 72
133
= .
1728
Therefore

Cov 𝑌1 , 𝑌2
𝜌 𝑌1 , 𝑌2 = = 0.2710 ⋯
Var 𝑌1 Var(𝑌2 )

(ii) Clearly, for a fixed 𝑥1 ∈ 0, 1 ,

𝑓𝑋2 ,𝑋3 𝑋1 𝑥2 , 𝑥3 𝑥1 = 𝑐1 𝑥1 𝑓𝑋1 ,𝑋2 ,𝑋3 𝑥1 , 𝑥2 , 𝑥3

Dept. of Mathematics and Statistics Indian Institute of Technology, Kanpur 7

NPTEL- Probability and Distributions

𝑐2 (𝑥 1 )
, if 0 < 𝑥3 < 𝑥2 < 𝑥1
= 𝑥2 .
0, otherwise

Since
∞ ∞

𝑓𝑋2 ,𝑋3 |𝑋1 𝑥2 , 𝑥3 𝑥1 𝑑𝑥2 𝑑𝑥3 = 1,

−∞ −∞

we have
𝑥1 𝑥2
1
𝑐2 𝑥1 𝑑𝑥 𝑑𝑥 = 1,
𝑥2 3 2
0 0

1
i. e., 𝑐2 𝑥1 = .
𝑥1

Also

𝐸(𝑌 𝑋1 = 𝑥1 ) = 𝐸 𝑋1 𝑋2 𝑋3 𝑋1 = 𝑥1

= 𝑥1 𝐸(𝑋2 𝑋3 𝑋1 = 𝑥1 )
𝑥1 𝑥2
1
= 𝑥1 𝑥2 𝑥3 𝑑𝑥 𝑑𝑥
𝑥1 𝑥2 3 2
0 0

𝑥13
= .
6
𝐸(𝑌 2 𝑋1 = 𝑥1 ) = 𝐸 𝑋12 𝑋22 𝑋32 𝑋1 = 𝑥1

= 𝑥12 𝐸(𝑋22 𝑋32 𝑋1 = 𝑥1 )

𝑥1 𝑥2
1
= 𝑥12 𝑥22 𝑥32 𝑑𝑥 𝑑𝑥
𝑥1 𝑥2 3 2
0 0

𝑥16
= .
15
Therefore

Var(𝑌 𝑋1 = 𝑥1 ) =𝐸(𝑌 2 𝑋1 = 𝑥1 ) − (𝐸(𝑌 𝑋1 = 𝑥1 ))2

Dept. of Mathematics and Statistics Indian Institute of Technology, Kanpur 8

NPTEL- Probability and Distributions

𝑥16 𝑥16
= −
15 36
7 6
= 𝑥 . ▄
180 1

6.6 JOINT MOMENT GENERATING FUNCTION

Let 𝑋 = (𝑋1 , … , 𝑋𝑝 ) be a 𝑝-dimensional random vector defined on a probability space
𝑝 𝑝
𝛺, ℱ, 𝑃 . Let 𝐴 = 𝑡 = (𝑡1 , 𝑡2 , … , 𝑡𝑝 ∈ ℝ𝑝 : 𝐸 𝑒 𝑖=1 𝑡 𝑖 𝑋 𝑖 =𝐸 𝑒 𝑖=1 𝑡 𝑖 𝑋 𝑖 is finite } .
Define the function 𝑀𝑋 ∶ 𝐴 → ℝ by
𝑝
𝑀𝑋 𝑡 = 𝐸 𝑒 𝑖=1 𝑡 𝑖 𝑋 𝑖 , 𝑡 = (𝑡1 , 𝑡2 , … , 𝑡𝑝 ) ∈ 𝐴. (6.1)

Definition 6.1

(i) The function 𝑀𝑋 ∶ 𝐴 → ℝ , defined by (6.1), is called the joint moment generating
function (m.g.f.) of random vector 𝑋.
(ii) We say that the joint m.g.f. of 𝑋 exists if it is finite in a rectangle −𝑎, 𝑎 ⊆ ℝ𝑝 , for
some 𝑎 = (𝑎1 , 𝑎2 , … , 𝑎𝑝 ) ∈ ℝ𝑝 ; here −𝑎 = (−𝑎1 , −𝑎2 , … , −𝑎𝑝 ) and −𝑎, 𝑎 = {𝑡 ∈
ℝ𝑝 : −𝑎𝑖 < 𝑡𝑖 < 𝑎𝑖 , 𝑖 = 1, 2, … , 𝑝}. ▄

As in the one- dimensional case many properties of probability distribution of 𝑋 can be

studied through joint m.g.f. of 𝑋 . Some of the results, which may be useful in this
direction, are provided below without their proofs. Note that 𝑀X 0 = 1. Also if
𝑋1 , … , 𝑋𝑝 are independent then
𝑝 𝑝 𝑝
𝑝
𝑀𝑋 𝑡 = 𝐸 𝑒 𝑖=1 𝑡 𝑖 𝑋 𝑖 =𝐸 𝑒 𝑡𝑖 𝑋𝑖 = 𝐸(𝑒 𝑡𝑖 𝑋 𝑖 ) = 𝑀𝑋 𝑖 𝑡𝑖 , 𝑡 ∈ ℝ𝑝 .
𝑖=1 𝑖=1 𝑖=1

Theorem 6.1

Suppose that 𝑀𝑋 𝑡 exists in a rectangle −𝑎, 𝑎 ⊆ ℝ𝑝 . Then 𝑀𝑋 𝑡 possesses partial

derivatives of all orders in −𝑎, 𝑎 . Furthermore, for positive integers 𝑘1 , … , 𝑘𝑝 ,

Dept. of Mathematics and Statistics Indian Institute of Technology, Kanpur 9

NPTEL- Probability and Distributions

𝑘 𝑘 𝑘 𝜕 𝑘 1 +𝑘 2 +𝑘 3 ⋯+𝑘 𝑝
𝐸 𝑋1 1 𝑋2 2 ⋯ 𝑋𝑝 𝑝 = 𝑘 𝑘 𝑀𝑋 𝑡 .
𝜕𝑡1 1 ⋯ 𝜕𝑡𝑝 𝑝
(𝑡 1 ,𝑡 2 ,…,𝑡 𝑝 )=(0,…,0)

Under the assumptions of Theorem 6.1, note that, for 𝜓𝑋 𝑡 = ln 𝑀𝑋 𝑡 , 𝑡 ∈ 𝐴,

𝜕 𝜕
𝐸(𝑋𝑖 ) = 𝑀 𝑡 = 𝜓 𝑡 , 𝑖 = 1, … , 𝑝
𝜕𝑡𝑖 𝑋 𝑡=0
𝜕𝑡𝑖 𝑋 𝑡=0

𝑚 𝜕𝑚
𝐸 𝑋𝑖 = 𝑀 𝑡 , 𝑖 = 1, … , 𝑝
𝜕𝑡𝑖 𝑚 𝑋 𝑡=0

2
𝜕2 𝜕
Var 𝑋𝑖 = 𝑀 𝑡 − 𝑀 𝑡
𝜕𝑡𝑖 2 𝑋 𝑡=0
𝜕𝑡𝑖 𝑋 𝑡=0

𝜕2
= 𝜓 𝑡 , 𝑖 = 1, … , 𝑝,
𝜕𝑡𝑖 2 𝑋 𝑡=0

and, for 𝑖, 𝑗 ∈ 1, … , 𝑝 , 𝑖 ≠ 𝑗,

Cov 𝑋𝑖 , 𝑋𝑗 = 𝐸(𝑋𝑖 𝑋𝑗 ) − 𝐸(𝑋𝑖 )𝐸 𝑋𝑗

𝜕2 𝜕 𝜕
= 𝑀 𝑡 − 𝑀 𝑡 𝑀 𝑡
𝜕𝑡𝑖 𝜕𝑡𝑗 𝑋 𝜕𝑡𝑖 𝑋 𝑡=0
𝜕𝑡𝑗 𝑋
𝑡=0 𝑡=0

𝜕2
= 𝜓 𝑡 .
𝜕𝑡𝑖 𝜕𝑡𝑗 𝑋 𝑡=0

Also note that

𝑀𝑋 0, … ,0, 𝑡𝑖 , 0, … ,0 = 𝐸(𝑒 𝑡𝑖 𝑋 𝑖 ) = 𝑀𝑋 𝑖 𝑡𝑖 , 𝑖 = 1,2, … , 𝑝.

and 𝑀𝑋 0, … ,0, 𝑡𝑖 , 0, … ,0, 𝑡𝑗 , 0, … ,0 = 𝐸(𝑒 𝑡𝑖 𝑋 𝑖 +𝑡 𝑗 𝑋 𝑗 ) = 𝑀𝑋 𝑖 ,𝑋 𝑗 𝑡𝑖 , 𝑡𝑗 , 𝑖, 𝑗 ∈ 1, … , 𝑝 ,

provided the involved expectations are finite.

Dept. of Mathematics and Statistics Indian Institute of Technology, Kanpur 10