Expected Length of Coding Scheme Suppose that several information sources genera

Question

Expected Length of Coding Scheme

Suppose that several information sources generate symbols at random from a five-letter alphabet {A, B, C, D, E} with different probabilities. We will try different encoding schemes for encoding these symbols in binary. Given the probabilities and an encoding scheme, you will compute the expected length of the encoding of n letters generated by the information source. You may use any rigorous method of analysis you like, but must show your work and justify your answer.

(a) Suppose that the symbols occur with probabilities Pr[A] =0.3, Pr[B] = 0.3, Pr[C] = 0.2, Pr[D] = 0.1, and Pr[E] = 0.1, and the coding scheme encodes these symbols into binary codes as follows:

A 001

B 010

C 011

D 100

E 101

What is the expected length of the encoding for these probabilities and encoding?

(b) Now suppose that the symbols occur with the same probabilities Pr[A] =0.3 , Pr[B] = 0.3, Pr[C] = 0.2, Pr[D] = 0.1, and Pr[E] = 0.1, but we have a different encoding scheme:

A 0

B 10

C 110

D 1110

E 1111

What is the expected length of the encoding for these probabilities and encoding? Which encoding scheme is better for these probabilities?

(c) Now consider a different information system that generates symbols with probabilities Pr[A] =0.5, Pr[B] = 0.2, Pr[C] = 0.2, Pr[D] = 0.05, and Pr[E] = 0.05. We will use the same encoding scheme:

A 0

B 10

C 110

D 1110

E 1111

What is the expected length of the encoding for these probabilities and encoding?

Explanation / Answer

(a) Suppose that the symbols occur with probabilities Pr[A] =0.3, Pr[B] = 0.3, Pr[C] = 0.2, Pr[D] = 0.1, and Pr[E] = 0.1, and the coding scheme encodes these symbols into binary codes as follows:

A 001

B 010

C 011

D 100

E 101

Answer : Considering the provided binary codes of symbols A,B,C,D,E as Huffman code, We have a formula for calculation of expected codeword length i.e. :

a1*pr1 + a2*pr2 + a3*pr3 +...........+an*prn

Let's cconvert symbols binary codes into their decimal equivalents:

code for A is 001, so its decimal equivalent will be 1, similarly decimal equivalent for other symbols are as follows:

B(010) = 2(decimal equivalent)

C(011) = 3(decimal equivalent)

D(100) = 4(decimal equivalent)

E(101) = 5(decimal equivalent)

so, putting the values in formula we get :

expected length of code is = 0.3*1 + 0.3*2 + 0.2*3 + 0.1*4 + 0.1*5

= 0.3 + 0.6 + 0.6 + 0.4 + 0.5

= 2.4

----------------------------------------------------------------------------------------------------------------------------------------

(b) Now suppose that the symbols occur with the same probabilities Pr[A] =0.3 , Pr[B] = 0.3, Pr[C] = 0.2, Pr[D] = 0.1, and Pr[E] = 0.1, but we have a different encoding scheme:

A 0

B 10

C 110

D 1110

E 1111

Answer : Using the formula just like in above question

expeted code length = a1*pr1 + a2*pr2 + a3*pr3 +...........+an*prn

We need to convert binary codes into respective decimal equivalent:  

A 0 = 0 (decimal equivalent )

B 10 = 2 (decimal equivalent )

C 110 = 6 (decimal equivalent )

D 1110 = 14 (decimal equivalent )

E 1111 = 15 (decimal equivalent )

Next, putting values in the formula:  

Expected code length = 0.3* 0 + 0.3* 2 + 0.2* 6 + 0.1* 14 + 0.1* 15

= 4.7

--------------------------------------------------------------------------------------------------------------------------------

(c) Now consider a different information system that generates symbols with probabilities Pr[A] =0.5, Pr[B] = 0.2, Pr[C] = 0.2, Pr[D] = 0.05, and Pr[E] = 0.05. We will use the same encoding scheme:

A 0

B 10

C 110

D 1110

E 1111

Answer: We are following the approach just like we did in question 1 and 2

expeted code length = a1*pr1 + a2*pr2 + a3*pr3 +...........+an*prn

We need to convert binary codes into respective decimal equivalent:  

A 0 = 0 (decimal equivalent )

B 10 = 2 (decimal equivalent )

C 110 = 6 (decimal equivalent )

D 1110 = 14 (decimal equivalent )

E 1111 = 15 (decimal equivalent )

Next, putting values in the formula :

expected code length = 0.5 * 0 + 0.2 * 2 + 0.2 * 6 + 0.05 * 14 + 0.05 * 15

= 0 + 0.4 + 1.2 + 0.7 + 0.75

= 3.05

Related Questions

Navigate

• Browse (All)
• Browse E
• Subjects

Integrity-first tutoring: explanations and feedback only — we do not complete graded work. Learn more.