DeiT

E435881

DeiT is a family of data-efficient vision transformer models designed for image classification with reduced training data requirements and strong performance.

All labels observed (9)

How this entity was disambiguated

Statements (48)

Predicate Object
instanceOf deep learning model ⓘ
image classification model ⓘ
vision transformer model family ⓘ
abbreviationFor Data-efficient Image Transformers ⓘ
linked to: DeiT
availableIn Hugging Face Transformers ⓘ
timm library ⓘ
basedOn ViT architecture ⓘ
Vision Transformer ⓘ
linked to: ViT
codeRepository https://github.com/facebookresearch/deit ⓘ
contribution reduced data requirements for vision transformers ⓘ
showed transformers can be trained from scratch on ImageNet-1k only ⓘ
designedFor image classification ⓘ
developedBy Facebook AI Research ⓘ
linked to: Meta AI

Meta AI ⓘ
doesNotRequire ImageNet-21k pretraining ⓘ
JFT pretraining ⓘ
firstAuthor Hugo Touvron ⓘ
fullName Data-efficient Image Transformers ⓘ
linked to: DeiT
hasAuthor Alexandre Sablayrolles ⓘ
Gabriel Synnaeve ⓘ
Herve Jegou ⓘ
Hugo Touvron ⓘ
Matthieu Cord ⓘ
hasProperty competitive accuracy on ImageNet ⓘ
reduced training data requirements ⓘ
strong performance with limited data ⓘ
hasVariant DeiT-B ⓘ
linked to: DeiT

DeiT-B distilled ⓘ
linked to: DeiT

DeiT-S ⓘ
linked to: DeiT

DeiT-S distilled ⓘ
linked to: DeiT

DeiT-Ti ⓘ
linked to: DeiT

DeiT-Ti distilled ⓘ
linked to: DeiT
implementedIn PyTorch ⓘ
inputType 2D images ⓘ
introducedDistillationMethod token-based knowledge distillation ⓘ
license Apache-2.0 (for official code release) ⓘ
optimizedFor data efficiency ⓘ
paperTitle Training data-efficient image transformers & distillation through attention ⓘ
linked to: DeiT
publicationYear 2020 ⓘ
publishedAs research paper ⓘ
task supervised image classification ⓘ
teacherModelType convolutional neural network ⓘ
trainingDataset ImageNet-1k ⓘ
linked to: ImageNet
uses class token ⓘ
distillation token ⓘ
knowledge distillation from CNN teacher ⓘ
patch embedding ⓘ
transformer encoder ⓘ

How these facts were elicited

Referenced by (11)

Full triples — surface form annotated when it differs from this entity's canonical label.

ViT → hasVariant → DeiT ⓘ
DeiT → fullName → Data-efficient Image Transformers ⓘ
linked to: DeiT
DeiT → abbreviationFor → Data-efficient Image Transformers ⓘ
linked to: DeiT
DeiT → paperTitle → Training data-efficient image transformers & distillation through attention ⓘ
linked to: DeiT
DeiT → hasVariant → DeiT-Ti ⓘ
linked to: DeiT
DeiT → hasVariant → DeiT-S ⓘ
linked to: DeiT
DeiT → hasVariant → DeiT-B ⓘ
linked to: DeiT
DeiT → hasVariant → DeiT-Ti distilled ⓘ
linked to: DeiT
DeiT → hasVariant → DeiT-S distilled ⓘ
linked to: DeiT
DeiT → hasVariant → DeiT-B distilled ⓘ
linked to: DeiT