Information Theory and Coding

*************************************************************

ITC TEST 2 -2009

PART A:

1. Given an eight symbol source with probabilities

P={0.25, 0.20, 0.15, 0.15, 0.10, 0.05, 0.05, 0.05}

 Construct two binary Huffman codes as directed.

(i)                  Place the composite symbol “as low as you can”

(ii)                Move the composite symbol  “as high as possible”.

In each case determine the variance of the word lengths and comment on the results.                                                                                             -10-

      2. (a)  A Binary Symmetric Channel has an error probability of 0.1. Find its capacities after deriving relevant relations.                                                                   -5-

      

           (b) Explain Binary Erasure Channel and find its channel capacity.       -5-

PART B:

3        (a) For a channel whose matrix is given below

                                                     Y

                                            0.6  0.2  0.2                                                         -8-

P[Y/X] = X   0.2  0.6  0.2

0.2    0.2  0.6

Find I(X,Y) and channel capacity given the input symbols occur with equal probability.

    

            (b) Define Mutual Information                                                                 -2-

4. Consider a source S = {s₁, s₂} with probabilities ¾ and ¼  respectively. Obtain Shanon-Fano code for source S. it’s 2^nd and 3^rd extensions. Calculate efficiencies for each case.                                                                                                                        -10-

  

PART C

5. Explain all the properties of Mutual Information                                               -5-

6. What is a Binary Erasure Channel. Find the Channel Capacity of a Binary Erasure Channel.                                                                                                                  -5-

***************************************************************************