A Review on Transferability Estimation in Deep Transfer Learning

Yihao Xue, Rui Yang, Xiaohan Chen, Weibo Liu, Zidong Wang, Xiaohui Liu

Research output: Contribution to journalArticlepeer-review

Abstract

Deep transfer learning has become increasingly prevalent in various fields such as industry and medical science in recent years. To ensure the successful implementation of target tasks and improve the transfer performance, it is meaningful to prevent negative transfer. However, the dissimilarity between the data from source domain and target domain can pose challenges to transfer learning. Additionally, different transfer models exhibit significant variations in performance for target tasks, potentially leading to a negative transfer phenomenon. To mitigate the adverse effects of the above factors, transferability estimation methods are employed in this field to evaluate the transferability of the data and the models of various deep transfer learning methods. These methods ascertain transferability by incorporating mutual information between the data or models of the source domain and the target domain. This paper furnishes a comprehensive overview of four categories of transferability estimation methods in recent years. It employs qualitative analysis to evaluate various transferability estimation approaches, assisting researchers in selecting appropriate methods. Furthermore, this paper evaluates the open problems associated with transferability estimation methods, proposing potential emerging areas for further research. Lastly, the open-source datasets commonly used in transferability estimation studies are summarized in this study.

Original languageEnglish
Pages (from-to)1-21
Number of pages21
JournalIEEE Transactions on Artificial Intelligence
DOIs
Publication statusAccepted/In press - 2024
Externally publishedYes

Keywords

  • Brain modeling
  • Computational modeling
  • Data models
  • Deep transfer learning
  • Estimation
  • negative transfer
  • Reviews
  • Task analysis
  • Transfer learning
  • transfer performance
  • transferability estimation

Fingerprint

Dive into the research topics of 'A Review on Transferability Estimation in Deep Transfer Learning'. Together they form a unique fingerprint.

Cite this