A Review on Transferability Estimation in Deep Transfer Learning

Abstract

Deep transfer learning has become increasingly prevalent in various fields such as industry and medical science in recent years. To ensure the successful implementation of target tasks and improve the transfer performance, it is meaningful to prevent negative transfer. However, the dissimilarity between the data from source domain and target domain can pose challenges to transfer learning. Additionally, different transfer models exhibit significant variations in performance for target tasks, potentially leading to a negative transfer phenomenon. To mitigate the adverse effects of the above factors, transferability estimation methods are employed in this field to evaluate the transferability of the data and the models of various deep transfer learning methods. These methods ascertain transferability by incorporating mutual information between the data or models of the source domain and the target domain. This paper furnishes a comprehensive overview of four categories of transferability estimation methods in recent years. It employs qualitative analysis to evaluate various transferability estimation approaches, assisting researchers in selecting appropriate methods. Furthermore, this paper evaluates the open problems associated with transferability estimation methods, proposing potential emerging areas for further research. Lastly, the open-source datasets commonly used in transferability estimation studies are summarized in this study.