[go: up one dir, main page]
Scribd
Upload
11 views1 page

ITC Assignment-1 Questions

The document is an assignment on Information Theory and Coding, covering various topics such as entropy, source and channel coding, and specific coding techniques like Huffman and Lempel-Ziv. It includes questions on calculating entropies, proving identities related to mutual information, and explaining concepts like the Weak Law of Large Numbers and Universal Source Coding. Additionally, it requires practical applications of coding techniques using given probabilities and sequences.

Uploaded by

seshadri2823
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
11 views1 page

ITC Assignment-1 Questions

The document is an assignment on Information Theory and Coding, covering various topics such as entropy, source and channel coding, and specific coding techniques like Huffman and Lempel-Ziv. It includes questions on calculating entropies, proving identities related to mutual information, and explaining concepts like the Weak Law of Large Numbers and Universal Source Coding. Additionally, it requires practical applications of coding techniques using given probabilities and sequences.

Uploaded by

seshadri2823
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 1

ITC Assignment-1

Q1) Explain the following Terms with respect to Information Theory and Coding.
i) Information & Its Units ii) Source Coding and Channel Coding iii)Prefix free coding
iv) Rooted tree with Probabilities v)Tunstall Coding
Q2) i) Consider that two sources emot messages x1,x2,x3,x4 and y1,y2,y3 with the joint
probabilities P(X,Y) as shown in the matrix form:

Calculate the entropies H(X), H(Y), H (X, Y), H(X/Y), H(Y/X) and
Mutual Information I (X; Y)
ii) Define Entropy and then Find the condition for Maximum Entropy Hmax for M=2
and also estimate Hmax for M messages
Q3) i) Define Joint and Conditional Entropies and then Prove the following Identites
H(X, Y)=H(X)+H(Y/X) and H(X,Y)=H(Y)+H(X/Y)
ii) Define Average Mitual Information and then Prove the following Identites
I(X ;Y)= H(X)-H(X/Y)=H(Y)-H(Y/X)=H(X)+H(Y)-H(X,Y)
Q4) The source of information A generates the symbols { A1. A2, A3, A4, A5, A6, A7} with
the following probabilities {0.2 0.3 0.11 0.16 0.18 0.05}. Explain the code for source
symbols usimg Shannon fano encoder and find its code efficiency.
Q5) Construct the Huffman code with minimum code variance for the following probabilities
and also determine the code variance and code efficiency :{0.25 0.25 0.125, 0.125, 0.125,
0.0625, 0.0625}
Q6) Explain the Following with respect to Information Theory and Coding
a) AEP and its Properties
b) Typical Set and its Properties
Q7) Explain the Following with respect to Information Theory and Coding
a) Weak Law of Large Numbers(WLLN)
b) Chebyschev Inequality
Q8) Explain the steps in Lempel-Ziv (LZ) Algorithm and investigate how the given data
sequence “ A A B A B B B A B A A B A B B B A B B A B B ” is encoded using Lempel-Ziv
coding.
1.<0,0,C(c)>
Q9) Define Search Buffer and Look-Ahead Buffer in the context of LZ77 2.<0,0,C(a)>
coding. Given: 3. <0,0,C(b)>
 Search buffer size (w) = 13 4. <0,0,C(r)>
 Look-ahead buffer size (s) = 7 5. <3,1,C(c)>
 Maximum match length (L) = 6 6. <2,1,C(d)>
Analyze the sequence of triplets shown and decode them to obtain the original 7 .<7,4,C(r)>
string 8. <3,5,C(d)>

Q10) a) What is Universal Source Coding? Explain its purpose with examples?
b) Explain Noisy Coding Theorem and Channel Capacity of Gaussian Channel

You might also like