AI GitHub
Grounding DINO for text-guided object detection
Find image regions that match natural-language phrases.
Key Takeaways
- The model combines visual input and text to predict boxes associated with concepts.
- Use a compatible model checkpoint and inspect the text prompts before evaluation.
- Try two related nouns and one absent object on the same image.
- An open vocabulary still reflects training and prompt wording.
How it works
The model combines visual input and text to predict boxes associated with concepts. Thresholds affect which boxes and phrase matches survive postprocessing.
What you need
Use a compatible model checkpoint and inspect the text prompts before evaluation. Keep phrase-to-box associations and scores when reviewing predicted detections.
What to try first
Try two related nouns and one absent object on the same image. Compare resulting boxes at a few thresholds and inspect false positives.
Put two categories with similar appearance in a scene and describe both. Inspect which words and boxes correspond instead of treating any returned box as a match.
Where to look for failures
An open vocabulary still reflects training and prompt wording. Evaluate phrases in your domain and use masks or human review when boxes are insufficient.
Related Franklin pages
Choose between text-guided boxes, prompted masks and downstream tracking. Preserve a few ambiguous images and videos so each stage can be evaluated independently.
- Connect detection outputs to annotation, tracking and downstream analysis utilities. Supervision for processing computer vision detections
- Estimate scene depth from a single image with published checkpoints. Depth Anything V2 for monocular depth estimation
For an overview of the types of predictions vision systems produce, see Franklin’s glossary. Computer vision
Project documentation
Try it for your workflow
Explore the project’s documentation for setup and usage, or save this page to a collection for later.
Read the project documentation ↗