Neural networks 1

• Home
• About the Course
• Credits
• Lecture and Exercise Notes

Conditions for Obtaining Credit:

1. Mandatory: Completion of a project and its personal presentation during the lecture (in the final week 13.5.2025 or earlier).
2. Mandatory: Oral examination on the theory.
3. Optional (bonus points): Active participation in lessons and completion of exercises.

Active Participation in Lectures and Practical Sessions, Solving Tasks

• Attendance is not mandatory, but it is recommended and will contribute to a better grade.
• Active participation means focusing on the class, being involved in solving problems, and communicating.
• Occasionally, some bonus tasks will be assigned for extra points (typically as homework).
• Grading of attendance and bonus points: 1 point for active participation in one class, with a maximum of 26 points possible.
  • For an A grade based on attendance: at least 20 out of 26 points
  • For a B grade based on attendance: at least 13 out of 26 points
  • If you have fewer than 13 points (or if you prefer), your final grade will be calculated purely based on the project and a more thorough examination.

Project Development and Presentation

During this course, you will work independently on a project, either individually or, in the case of a more extensive topic, in pairs or triples. You will then present this project during a lesson. The goal of the project is to apply one of the studied models to a real-world data problem and to experiment with various methods and techniques.

• What does the project involve?
  • Choice of task and dataset: Select a suitable task and a corresponding dataset. For example, it can be classification, regression, working with image or sequential data – whichever interests you. If you are unsure, I will be happy to suggest a task.
  • Data preprocessing: It is crucial to prepare the data thoroughly for training the models – this can include cleaning, vectorization, augmentation, etc.
  • Model selection and application: Use one or more of the covered models that are suitable for the chosen problem.
  • Experimentation:
    • Try different approaches (e.g., various models, different variations of one model, different architectures or parameter choices, pre-trained models). You can also apply techniques to improve training (e.g., regularization, dropout, data augmentation, ...). The experimentation process is important. How (and to what extent) you experiment is up to you, but I am happy to give advice.
    • Compare the results of multiple models or approaches.
  • Evaluation: Assess the results of your experiment – how well does your model (or models) perform? Which techniques had the greatest impact and how? What was the biggest challenge for you? Did you learn anything new?
  • Tip: If you have already worked or are currently working on some project related to neural networks (e.g., as part of a bachelor’s thesis or any other research), you can use it for this course upon agreement.
• Important deadlines:
  • April 15 – May 1, 2025: Selection of the task and its approval.
  • May 13, 2025: Presentations of the completed projects to the class. Along with the presentation, you will submit the annotated source code (a notebook is ideal), input data, output visualizations, and possibly slides (materials) for your presentation. If needed, you can present earlier by prior arrangement.
  • Additional notes: If you do not finish your project on time, you will present the current status during the week of May 12 – 16, 2025 and then submit a more detailed project report by September 15, 2025 at the latest. The final defense of the project will then take place during an individual consultation (ideally together with the examination). Similarly, if the project and its presentation have significant shortcomings or are insufficient in scope, I may ask you to supplement them, and then submit and present these additions later (by September 15, 2025 at the latest).
• The specific grade (A–E) will depend on the scope and quality of the project. Whether the project was submitted on time will also be taken into consideration.

FAQ about Projects

• What structure should the final presentation have?
  • A description of the task you decided to solve.
  • A description of the data (the nature of the data, features, number of samples, etc.) and their source (or how you obtained the data).
  • How you preprocessed the data.
  • A description of the experiment – which models, techniques, and parameter settings you tried, and how.
  • Results of the experiment (appropriate visualization using tables, images, or graphs) + your own commentary.
• Where can I get data?
  • Kaggle: https://www.kaggle.com/datasets
  • UCI Machine Learning Repository: https://archive.ics.uci.edu/
  • KDnuggets: https://www.kdnuggets.com/datasets/index.html
  • GitHub: https://github.com/topics/machine-learning-datasets
  • Google Dataset Search: https://datasetsearch.research.google.com/
  • OpenML: https://www.openml.org/
  • Time series (e.g., currency rates, stock prices), and so on.
• Which models should I choose?
  You can select any of the models covered in the course. I particularly recommend:
  • A multilayer perceptron (MLP).
  • A simple neuron (e.g., perceptron, linear neuron). A neural network with one layer of neurons.
  • A competitive model. Kohonen map. RBF network.
  • A convolutional neural network (CNN).
• How do I compare models?
  • Depending on the task and model, use, for example, k-fold cross-validation or the average result of at least 10 runs. In tables, we typically compare speed (time, number of epochs), error size (MSE / classification error – on both the training and testing sets), and we are interested in both the mean and the variance/standard deviation of the results.
  • Architecture for a layered neural network – try various numbers of neurons, different numbers of layers, show results in table form, and evaluate them in your own words.
  • Other parameters – try different combinations of values, present the results in table form, and evaluate them in your own words.

Oral examination on the covered theory.

• A question or questions will be asked from a previously known list of topics (example question: “Perceptron: describe the model and its training algorithm, what tasks is it suitable for, what are its advantages and disadvantages?”).
• List of topics (questions) – will be updated
1. Basic concepts: machine learning, mathematical model of a neuron, neural network, multilayer neural network, deep vs. shallow model
2. Perceptron, representation of logical functions (logical threshold circuit), linear separability, perceptron learning algorithm (principle, variants, pitfalls)
3. Artificial neuron, activation function and its importance, when each function is suitable. Activation function in the hidden layer, in the output layer, for which tasks it is suitable or not, what data it requires.
4. Linear neuron and linear neural network, model description, learning algorithms (principle, comparison), which tasks they are suitable or not suitable for, what data they require, limitations. Relation to linear regression.
5. Neuron with continuous activation function, model description, training with gradient descent algorithms, which tasks it is suitable or not suitable for, limitations and problems.
6. Single-layer neural network – a brief description of the model and its training, which tasks it is suitable for, what data it requires, limitations.
7. Multilayer neural network and backpropagation (just the basic principle), its advantages and disadvantages, typical tasks, and data requirements. Architecture depending on the task type.
8. Multilayer neural networks – hyperparameters, training algorithms. Techniques for speeding up and improving learning. The model's ability to generalize and how to improve it.
9. The convolution operation and its significance in convolutional networks (and elsewhere). Convolutional layer.
10. Convolutional neural network – architecture, layers, applications, advantages, disadvantages. CNN training, transfer learning, techniques for improving training. Modern architectures.
11. Clustering – what it is, typical tasks, algorithms, their advantages and disadvantages.
12. Competitive model and Kohonen map – description of the models and their training, which tasks they are suitable or not suitable for, advantages and disadvantages.
• During the examination, I want to see whether you have a general understanding of the topic, how the models work, and what they are suitable for, and if you know the principle of the algorithms. I do not test specific mathematical formulas or technical details.