AI GitHub
AnimateDiff for adding motion to image diffusion workflows
Generate animations by combining compatible image-model components with motion modules.
Key Takeaways
- The approach attaches learned motion components to supported text-to-image models.
- Begin with a compatible image checkpoint and motion module.
- Use one base model and prompt, then compare two documented motion settings.
- Motion modules and checkpoints need compatible architectures.
How it works
The approach attaches learned motion components to supported text-to-image models. The image model contributes visual style while the motion module changes temporal behavior.
What you need
Begin with a compatible image checkpoint and motion module. Save the base generation settings so changes can be attributed to the temporal component.
What to try first
Use one base model and prompt, then compare two documented motion settings. Examine whether subject appearance holds across the whole sequence.
Try a fixed viewpoint with a single moving detail. It provides a simple baseline before asking the motion module for a complex moving camera.
Where to look for failures
Motion modules and checkpoints need compatible architectures. Match the versions in an example before troubleshooting a custom workflow.
Related Franklin pages
A reference frame, motion module and full graph affect output in different ways. Save the workflow and inspect early and late frames when testing a clip's continuity.
- Experiment with generating video using a compact representation of prior frames. FramePack for memory-aware video generation
- Build reusable generation graphs around models, samplers and conditioning inputs. ComfyUI for node-based image generation
For video assembled from authored HTML and motion graphics, see Franklin’s HyperFrames profile. HyperFrames
Project documentation
Try it for your workflow
Explore the project’s documentation for setup and usage, or save this page to a collection for later.
Read the project documentation ↗