Neural networks 2

• Home
• About the Course
• Credits
• Lecture and Exercise Notes

About the Course

We will get acquainted with various basic models of deep neural networks (basic model, convolutional neural networks, recurrent neural networks, chosen generative models, tranformers). We will learn how to implement the discussed models in Python using popular libraries and apply them to solve practical tasks.

The course builds on the course Neural Networks 1 (18NES1, 2+2 C, summer semester).

Expected Content of Practical Sessions

1. Introduction to Deep Learning: history, basic concepts. Frameworks (Keras, TensorFlow, PyTorch) and their usage
2. Fundamentals of Deep Neural Networks: architectures, activation functions, implementation and training on a sample dataset
3. Basic task types (classification, regression, time series prediction) – specifics and examples
4. Data for Deep Learning: acquisition, preprocessing, exploratory analysis, normalization, standardization, augmentation
5. Image classification: convolutional neural networks (CNN), principles, implementation, selected architectures
6. Training and tuning models: optimization, hyperparameter tuning, regularization, learning strategies, pretrained models and transfer learning
7. Advanced CNN applications: object detection, segmentation, encoder–decoder architectures
8. Modeling sequential data: time series, recurrent neural networks (RNN, LSTM, GRU)
9. Natural language processing: from RNNs to Transformers, practical examples (e.g., sentiment analysis)
10. Generative models: autoencoders, variational autoencoders, GANs and their applications
11. Student project presentations

Details are also available in KOS.

Conditions for Obtaining Credit

See the Credits tab.

Links (Machine Learning Courses at the Faculty of Nuclear Sciences and Physical Engineering)

• 18NES1 Neural Networks 1
  • 2nd year of Bachelor's degree, Summer Semester, KSI
  • Introduction to artificial neural networks. Emphasis on simple models and their understanding.
• 18NES2 Neural Networks 2
  • 3rd year of Bachelor's degree, Winter Semester, KSI
  • Deep neural networks. Emphasis on practical skills.
• U1USU Introduction to Machine Learning
  • 3rd year of Bachelor's degree, Winter Semester, KM
  • Basic models and methods of machine learning.
• U1SU2 Machine Learning 2
  • 1st year of Master's degree, Winter Semester, KM
  • Advanced models and methods of machine learning. Deep neural networks and other methods in depth :-).
• 18UIA1 Introduction to Advanced Algorithms
  • 1st year of Master's degree, (but also possible for BS), Winter Semester, KSI
  • Additional methods and algorithms for machine learning and artificial intelligence, focused (not only) on robotics.
• 18UIA2 Advanced Algorithms 2
  • 1st year of Master's degree, (but also possible for BS), Summer Semester, KSI
  • Further selected methods and algorithms for machine learning and artificial intelligence, focused (not only) on robotics.
• 18SC Softcomputing
  • 1st year NMS, Winter Semester, KSI
  • Fuzzy systems and related neural network models.
• 01AOM Applications of Optimization Methods
  • 2nd year of Master's degree, Winter Semester, KM
  • More about optimization methods (not only) for deep neural networks.
• 01NEUR1 Neural Networks and Their Applications 1
  • 1st year of Master's degree, Summer Semester, KM
  • Theoretical aspects of neural networks.
• 01NEUR2 Theoretical Foundations of Neural Networks
  • 2nd year of Master's degree, Winter Semester, KM
  • Theoretical aspects of neural networks.

References

[1] M. Nielson: Neural Networks and Deep Learning, 2019. available online. Github with examples. An inside look at deep learning.
[2] F. Chollet, M. Watson: Deep Learning with Python, Second Edition, 2021 (Third Edition – 2025). Github with examples. Also available in Czech. A detailed introduction to deep learning.
[3] A. Kapoor, A. Gulli, S. Pal: Deep Learning with TensorFlow and Keras – 3rd edition, 2022. Github with examples. More advanced methods of deep learning.
[4] Official Keras documentation, worked examples: Keras examples
[5] I. Goodfellow, Y. Bengio, A. Courville: Deep Learning, MIT Press, 2016. A classic textbook with a theoretical focus.
[6] Kaggle Notebooks. Further inspiration and source of examples.

Good Video Courses:

[1] https://www.youtube.com/playlist?list=PL2o3po04f3KZL0xq1PFjn3FUGl-uEiXEn A video course by František Voldřich in Czech based on M. Nielson’s book.
[2] https://ufal.mff.cuni.cz/courses/npfl138/2324-summer A course at Charles University on deep learning. Lecture and exercise recordings available (in CZ, EN), plus materials and example assignments.
[3] http://introtodeeplearning.com/ MIT course. Lecture recordings available.

Development Environment and Libraries

• Google Colab https://colab.research.google.com
  Advantage: Environment similar to JupyterLab, nothing to install or configure, (limited) GPU access, integrated AI features.
  Disadvantage: Requires a Google account, you must be online, the service is “beyond our control,” and there are various limitations.
• You can also work on your own computer in your preferred development environment; during the semester you will need to install the necessary libraries (Keras, TensorFlow, etc.).
  Sometimes a GPU can be useful, but for simpler tasks you can do without one.
• If you are new to Python, please go through our Python Programming Basics course materials https://gitlab.fjfi.cvut.cz/ksi/zpro-2024-public-en

Programs from Practical Sessions

They will be available on GitHub.

Consultations During the Semester

Consultations during the semester are possible by prior arrangement (by email or in person) on weekdays between 9:00 and 14:30, outside of class times and my other duties. The recommended consultation time is after the lecture/practical session. You can also send me your questions or comments by email. When emailing, please include the text "NES2" in the subject line.

Contact