Information Theory
(3-0-0-3)
CMPE Degree: This course is Not Applicable for the CMPE degree.
EE Degree: This course is Not Applicable for the EE degree.
Lab Hours: 0 supervised lab hours and 0 unsupervised lab hours.
Technical Interest Group(s) / Course Type(s): Telecommunications
Course Coordinator:
Prerequisites: ECE 3075
Catalog Description
To introduce the mathematical theory of communications. Emphasis will beplaced on Shannon's theorems and their use in the analysis and design of
communication systems
Textbook(s)
Elements of Information TheoryCourse Outcomes
Not Applicable
Strategic Performance Indicators (SPIs)
Not Applicable
Topical Outline
Entropy and Mutual Information Theory
-Joint Entropy, Conditional Entropy
-Data Processing Theorem
-Fano's Inequality
Asymptotic Equipartition Principle
-Typical Sequences
-Entropy, Source Coding and the AEP
-Joint Typicality (Neuhoff/Forney notes)
Entropy Rate
-Conditional Independence and Markov Chains
-Entropy Rate
Lossless Source Coding
-Kraft Inequality
-Shannon and Huffman Codes
-Shannon, Fano, Elias Codes
-Arithmetic Codes
-Lempel Ziv Codes
Channel Capacity
-Symmetric Channels
-Discrete Memoryless Channels and Their Capacity
-Arimoto-Blahut Algorithm
-Proof of the Channel Coding Theorem
-Converse of Channel Coding Theorem
Differential Entropy
-Entropy, Mutual Information, AEP for Continuous rv's
Gaussian Channel
-Capacity of AWGN, Bandlimited AWGN Channels
-Capacity of Nonwhite Channels: Water Filling
Rate Distortion Theory
-Quantization
-Rate Distortion Functions
-Vector Quantization
-Vector Quantization Gains
-Vector Quantization Design
Multiuser Information Theory (as time allows)
Information Theory and Statistics (as time allows)