A survey on self-supervised learning methods for domain adaptation in deep neural networks focusing on the optimization problems

Deep convolutional neural networks have been widely and successfully used for various computer vision tasks. The main bottleneck for developing these models has been the lack of large datasets labeled by human experts. Self-supervised learning approaches have been used to deal with this challenge and allow developing models for domains with small labeled datasets. Another challenge for developing deep learning models is that their performance decreases when deployed on a target domain different from the source domain used for model training. Given a model trained on a source domain, domain adaptation refers to the methods used for adjusting a model or its output such that when the model is applied to a target domain, it achieves higher performance. This paper reviews the most commonly used self-supervised learning approaches and highlights their utility for domain adaptation.