Следующий поток онлайн-школы: осень 2024. Предварительная регистрация и подробности здесь:
Подпишитесь на телеграм-канал с анонсами стримов:
Плейлист "AI: от основ до трансформеров": • AI: от основ до трансформеров DeiT III: Revenge of the ViT Touvron, Hugo, Matthieu Cord, and Hervé Jégou. "Deit iii: Revenge of the vit." European Conference on Computer Vision. Cham: Springer Nature Switzerland, 2022.
Knowledge Distillation Hinton, Geoffrey, Oriol Vinyals, and Jeff Dean. "Distilling the knowledge in a neural network." arXiv preprint arXiv:1503.02531 (2015).
BEiT Bao, Hangbo, et al. "Beit: Bert pre-training of image transformers." arXiv preprint arXiv:2106.08254 (2021).
DeiT Touvron, Hugo, et al. "Training data-efficient image transformers & distillation through attention." International conference on machine learning. PMLR, 2021.
LayerScale Touvron, Hugo, et al. "Going deeper with image transformers." Proceedings of the IEEE/CVF international conference on computer vision. 2021.
BYOL Лекция:
https://youtube.com/live/9ld7I_R1o9Y Grill, Jean-Bastien, et al. "Bootstrap your own latent-a new approach to self-supervised learning." Advances in neural information processing systems 33 (2020): 21271-21284.
