Neural networks 1

• Home
• About the Course
• Credits
• Lecture and Exercise Notes

About the Course

• This course is a complete introduction to the world of artificial neural networks.
• We will gradually learn the basic neural network models: a single neuron, models with a single layer of neurons, and models with multiple layers of neurons (multilayer perceptron, convolutional neural network).
• Our goal is to understand how artificial neural networks work internally, how they behave in various situations and why, and how to apply them correctly to solve different types of tasks.

Expected Content of Lectures and Practical Sessions

1. Introduction to Machine Learning and Artificial Neural Networks. History, biological motivation.
2. Artificial Neuron
   • Perceptron with a step transfer function: Model description, threshold logic circuit, and realization of logical functions. Linear separability. Learning algorithms for perceptron. Examples.
   • Linear Neuron: Model description, learning algorithms, linear neural network, connection with linear regression, linear classification. Examples.
   • Neurons with a continuous transfer function: Model description, learning algorithms. Examples.
3. Single-Layer Perceptron Network. Model description, learning algorithms, matrix representation. Examples.
4. Multi-layer Perceptron Network. Model description, transfer functions, task types, training data, backpropagation algorithm. Variants, analysis, practical applications, pitfalls. Examples.
5. Self-organization: Clustering, self-organizing neural networks, Kohonen maps, hybrid models.
6. Introduction to Deep Neural Networks
7. Convolutional Neural Networks. Architecture, layer types, training, practical examples.
8. Student Project Presentations.

Conditions for Obtaining Credit

See the Credits tab.

Links (Machine Learning Courses at the Faculty of Nuclear Sciences and Physical Engineering)

• 18NES1 Neural Networks 1
  • 2nd year of Bachelor's degree, Summer Semester, KSI
  • Introduction to artificial neural networks. Emphasis on simple models and their understanding.
• 18NES2 Neural Networks 2
  • 3rd year of Bachelor's degree, Winter Semester, KSI
  • Deep neural networks. Emphasis on practical skills.
• U1USU Introduction to Machine Learning
  • 3rd year of Bachelor's degree, Winter Semester, KM
  • Basic models and methods of machine learning.
• U1SU2 Machine Learning 2
  • 1st year of Master's degree, Winter Semester, KM
  • Advanced models and methods of machine learning. Deep neural networks and other methods in depth :-).
• 18UIA1 Introduction to Advanced Algorithms
  • 1st year of Master's degree, (but also possible for BS), Winter Semester, KSI
  • Additional methods and algorithms for machine learning and artificial intelligence, focused (not only) on robotics.
• 18UIA2 Advanced Algorithms 2
  • 1st year of Master's degree, (but also possible for BS), Summer Semester, KSI
  • Further selected methods and algorithms for machine learning and artificial intelligence, focused (not only) on robotics.
• 18SC Softcomputing
  • 1st year NMS, Winter Semester, KSI
  • Fuzzy systems and related neural network models.
• 01AOM Applications of Optimization Methods
  • 2nd year of Master's degree, Winter Semester, KM
  • More about optimization methods (not only) for deep neural networks.
• 01NEUR1 Neural Networks and Their Applications 1
  • 1st year of Master's degree, Summer Semester, KM
  • Theoretical aspects of neural networks.
• 01NEUR2 Theoretical Foundations of Neural Networks
  • 2nd year of Master's degree, Winter Semester, KM
  • Theoretical aspects of neural networks.

References

Classic Literature

[1] L.V. Fausett: Fundamentals of Neural Networks: Architectures, Algorithms and Applications, Prentice Hall, New Jersey, 1994.
[2] S. Haykin: Neural Networks, Macmillan, New York, 1994.
[3] R. Rojas: Neural Networks: A Systematic Introduction, Springer-Verlag, Berlin, 1996.

Modern Literature

[1] M. Nielson: Neural Networks and Deep Learning, 2019. available online. Github with examples. An inside look at deep learning.
[2] F. Chollet, M. Watson: Deep Learning with Python, Second Edition, 2021 (Third Edition – 2025). Github with examples. Also available in Czech. A detailed introduction to deep learning.
[3] A. Kapoor, A. Gulli, S. Pal: Deep Learning with TensorFlow and Keras – 3rd edition, 2022. Github with examples. More advanced methods of deep learning.
[4] Official Keras documentation, worked examples: Keras examples
[5] I. Goodfellow, Y. Bengio, A. Courville: Deep Learning, MIT Press, 2016. A classic textbook with a theoretical focus.
[6] Kaggle Notebooks. Further inspiration and source of examples.

Good Video Courses:

[1] https://www.youtube.com/playlist?list=PL2o3po04f3KZL0xq1PFjn3FUGl-uEiXEn A video course by František Voldřich in Czech based on M. Nielson’s book.
[2] https://ufal.mff.cuni.cz/courses/npfl138/2324-summer A course at Charles University on deep learning. Lecture and exercise recordings available (in CZ, EN), plus materials and example assignments.
[3] http://introtodeeplearning.com/ MIT course. Lecture recordings available.

Development Environment and Libraries

• Google Colab https://colab.research.google.com
  Advantage: Environment similar to JupyterLab, nothing to install or configure, (limited) GPU access, integrated AI features.
  Disadvantage: Requires a Google account, you must be online, the service is “beyond our control,” and there are various limitations.
• You can also work on your own computer in your preferred development environment; during the semester you will need to install the necessary libraries (Keras, TensorFlow, etc.).
  Sometimes a GPU can be useful, but for simpler tasks you can do without one.
• Faculty JupyterLab?
• If you are new to Python, please go through our Python Programming Basics course materials https://gitlab.fjfi.cvut.cz/ksi/zpro-2024-public-en

Programs from Practical Sessions

They will be available (currently on GitHub).

Consultations During the Semester

Consultations during the semester are possible by prior arrangement (by email or in person) on weekdays between 9:00 and 14:30, outside of class times and my other duties. The recommended consultation time is immediately after the lecture/practical session. You can also send me your questions or comments by email. When emailing, please include the text "NES1" in the subject line.

Contact