Video generation model
Generates short video clips from text or an image, keeping motion and objects consistent across frames.
What it is
Video generation models extend image generation through time. From a prompt or a starting image they produce a few seconds of footage with camera movement and moving subjects. Keeping people and objects consistent from frame to frame is the core difficulty.
How it works
Most are diffusion transformers that work on a compressed representation of the whole clip, treating space and time together. A video autoencoder squeezes frames into latents, the transformer denoises them under text guidance, and the decoder turns them back into pixels.
What it is good for
- Short marketing and social clips
- Animating a still image
- Storyboards and previsualization
- B-roll and visual effects drafts
Limits to know
- Clips are short and physics can look wrong
- Expensive to generate
- Deepfake and misuse risks
