Description:

Summary of probability theory;
Multivariate normal distribution;
Entropy and its application to coding;
Statistical tests: T-tests, goodness of fit tests, independence test;
Random processes - stacionarity;
Markov chains and limiting properties;
Queuing theory

Contents:

1. Summary of basic terms of probability theory
2. Random variables
3. Random vectors
4. Multivariate normal distribution
5. Entropy of discrete distributions
6. Application of entropy in coding theory
7. Entropy of continuous distributions
8. Summary of basic notions of statistics
9. Paired and Two-sample T-test,
10. Goodness of fit tests,
11. Independence testing, contingency tables
12. Estimation of PDF and CDF
13. Gaussian mixtures and EM algorithm
14. Random processes - stationarity
15. Random processes - examples (Gaussian, Poisson)
16. Memory-less distributions, exponential race
17. Discrete-time Markov chains - introduction
18. Discrete-time Markov chains - classification of states
19. Discrete-time Markov chains - stationarity
20. Discrete-time Markov chains - estimation of parameters
21 MCMC
22. Continuous time Markov chains - introduction
23. Continuous time Markov chains - Kolmogorov equations
24. Queuing theory, Little's theorem
25. Queuing systems M/M/1 and M/M/m
26. Queuing systems M/G/infinity

Seminar contents:

1. Review lesson: basics of probability
2. Random vectors, multivariate normal distribution
3. Entropy and coding theory
4. Entropy, mutual information
5. T-tests
6. Goodness of fit tests, independence test
7. Estimation of PDF and CDF
8. Random processes, Poisson process
9. Discrete-time Markov chains - stationarity
10. Discrete-time Markov chains - classification of states
11. Exponential race
12. Continuous-time Markov chains
13. Queuing theory

Recommended literature:

1. Cover, T. M. - Thomas, J. A. : Elements of Information Theory (2nd Edition). Wiley, 2006. ISBN 978-0-471-24195-9.
2. Durrett, R. : Essentials of Stochastic Processes. Springer, 1999. ISBN 978-0387988368.
3. Grimmett, G. - Stirzaker, D. : Probability and Random Processes (3rd Edition). Oxford University Press Inc., 2001. ISBN 978-0-19-857222-0.

Keywords:

Entropy; T-tests; Goodness of fit tests; Stochastic processes; Markov chains; Queuing theory;

Abbreviations used:

Semester:

• W ... winter semester (usually October - February)
• S ... spring semester (usually March - June)
• W,S ... both semesters

Mode of completion of the course:

• A ... Assessment (no grade is given to this course but credits are awarded. You will receive only P (Passed) of F (Failed) and number of credits)
• GA ... Graded Assessment (a grade is awarded for this course)
• EX ... Examination (a grade is awarded for this course)
• A, EX ... Examination (the award of Assessment is a precondition for taking the Examination in the given subject, a grade is awarded for this course)

Weekly load (hours per week):

• P ... lecture
• C ... seminar
• L ... laboratory
• R ... proseminar
• S ... seminar