On effects of Knowledge Distillation on Transfer Learning

This study combines transfer learning with knowledge distillation (TL+KD). It examines how matching a teacher during fine-tuning affects a student model’s generalization, qualitative behavior, and robustness.

Paper (arXiv)

BibTeX

@article{thapa2022knowledge,
  title   = {On Effects of Knowledge Distillation on Transfer Learning},
  author  = {Thapa, Sushil},
  journal = {arXiv preprint arXiv:2210.09668},
  year    = {2022}
}