Vision transformer and image classifier
Recognizes what is in an image by treating it as a grid of patches, the backbone of modern computer vision.
What it is
Image classifiers label what a picture shows. Vision transformers (ViTs) brought the transformer design to images and now underpin classification, the image side of VLMs and many detection models. Self-supervised versions learn strong features without labels.
How it works
The image is cut into small square patches, each flattened into a vector, like words in a sentence. A transformer relates the patches to each other, and a final layer outputs class scores or a general-purpose image embedding.
What it is good for
- Classifying photos and products
- Quality inspection in factories
- Medical image triage
- Feature extraction for other vision tasks
Limits to know
- Needs a lot of data or pre-training
- Can latch onto background cues
- Does not locate objects without a detection head
