Information Theory

Lectures

Ø Introduction (Lecture)

Ø Basic Notions and Inequalities (Lecture) (printable PDF) (Seminar 1) (Seminar 2)

Ø Asymptotic Equipartition Property (Lecture) (printable PDF) ( Seminar 3-4 )

Ø Stochastic Processes (Lecture) (printable PDF) ( Seminar 3-4 )

Ø Data Compression (PDF) (printable PDF); algorithm for Huffman code (PDF) ( Seminar 5 )

Ø Channel Capacity (PDF) (printable PDF)

Ø Differential Entropy (PDF) (printable PDF) ( Seminar 7 )

Ø Gaussian Channel (PDF) (printable PDF) ( Seminar 8 )

Ø Universal Source Coding (PDF) (printable PDF)

Ø Coding Theory (PDF) (printable PDF) ( Seminars - Code Theory )

Programming examples for seminars

Bibliography

1. Thomas M. Cover, Joy A. Thomas, Elements of Information Theory, 2nd edition, Wiley, 2006.

2. David J.C. MacKay, Information Theory, Inference, and Learning Algorithms, Cambridge University Press, 2003.

3. Robert M. Gray, Entropy and Information Theory, Springer, 2009

4. D. A. Huffman, A method for the construction of minimum redundancy codes, Proc. IRE, 40: 1098-1101,1952

Homeworks

Click here