Description:

The aim of the course "Neural Networks 2" is to acquaint students with basic models of deep neural networks and teach them how to apply these models and methods to solve practical tasks.

Seminar contents:

The exercises will focus on experimenting with various deep learning models using popular frameworks (such as TensorFlow or PyTorch) on practical tasks (processing image and sequential data, object detection, segmentation, etc.). Students will gain experience in analyzing results and learn about practical aspects of model implementation and tuning, which will help them better understand deep learning.



1. Introduction to Deep Learning: History and Basic Concepts. Existing Frameworks for Deep Learning. Basic Work with TensorFlow or PyTorch. Creating a Simple Neural Network with Numerical Data.
2. Deep Neural Networks: Architectures and Activation Functions. Implementing and Training a Deep Neural Network on the MNIST Dataset.
3. Introduction to Solving Basic Types of Tasks (Classification, Regression, Time Series Prediction). Specifics of Each Type of Task.
4. Convolutional Neural Networks: Basics and Principles. Classification Tasks. Architectures of Convolutional Neural Networks.
5. Deep Learning and Data. Acquisition, Preparation, and Processing of Data. Normalization and Standardization. Data Augmentation.
6. Algorithms for Deep Neural Network Training, Hyperparameter Optimization and Tuning (Grid Search, Random Search, Bayesian Optimization), Regularization Techniques for Deep Neural Networks.
7. Pre-training and Fine-tuning of Deep Neural Networks. Transfer Learning.
8. Recurrent Neural Networks and Sequential Data Processing.
9. Architectures of Recurrent Neural Networks.
10-11. Convolutional Network Architectures for Object Detection and Segmentation.
12. Autoencoders: Principles and Applications (Denoising, Dimensionality Reduction).
13. Gentle introduction to Other Neural Network Models (Generative Models, Transformers, Reinforcement Learning).

Recommended literature:

[1] Ian Goodfellow, Yoshua Bengio, Aaron Courville: Deep Learning, 2016, MIT Press
[2] Charu C. Aggarwal: Neural Networks and Deep Learning: A Textbook, 2018, Springer
[3]Ivan Vasilev, Daniel Slater: Python Deep Learning, 2019, Packt Publishing
[4] Andrew W. Trask: Grokking Deep Learning, 2019, Manning Publications

Keywords:

deep learning, convolutional neural networks, recurrent networks

Abbreviations used:

Semester:

• W ... winter semester (usually October - February)
• S ... spring semester (usually March - June)
• W,S ... both semesters

Mode of completion of the course:

• A ... Assessment (no grade is given to this course but credits are awarded. You will receive only P (Passed) of F (Failed) and number of credits)
• GA ... Graded Assessment (a grade is awarded for this course)
• EX ... Examination (a grade is awarded for this course)
• A, EX ... Examination (the award of Assessment is a precondition for taking the Examination in the given subject, a grade is awarded for this course)

Weekly load (hours per week):

• P ... lecture
• C ... seminar
• L ... laboratory
• R ... proseminar
• S ... seminar