Self-supervised Learning

With the development of deep learning, new tasks and datasets have emerged to train models. However, while data can be collected algorithmically without much difficulty, labeling is a very labor-intensive and expensive task. Therefore, there was a need to train models on unlabeled data. This gave rise to the self-supervised learning paradigm, which will be discussed in this course. Students will be introduced to both old and new approaches to pre-training on unlabeled data and will work with a variety of domains: from images and text to audio and graphs.