Information Theory

Lecturer: Priv.-Doz. Dr.-Ing. habil. Gholamreza Alirezaei

Lecture Index

  • Basics: entropy, Kullback-Leibler divergence, mutual information, capacity for absolutely-continuous distributions
  • Channels and their capacity: complex Gaussian, parallel, MMO, with feedback and memory, quantization and censoring
  • Rate distortion theory: achievability, computational aspects
  • Network information theory: multiple user channels, relay channels, broadcast channels, source coding with side information
  • information theory for neural networks (NN) and learning: single neuron classifier, information flow in NN, Boltzmann machines

Further information lecture: RWTHonline

Further information exercise: RWTHonline