Image diffusion model
Creates images by starting from noise and removing it step by step, guided by a text prompt.
What it is
Diffusion models are the main way AI generates images today. Given a prompt, they produce a new picture; given an image, they can edit, extend or restyle it. Variants handle inpainting, upscaling and control from sketches or poses.
How it works
During training the model learns to undo noise added to real images. To generate, it starts from pure noise and, over a number of steps, predicts and removes noise while a text encoder steers it toward the prompt. Latent diffusion does this in a compressed space, which makes it fast enough for consumer GPUs.
What it is good for
- Generating illustrations, product shots and concept art
- Editing parts of an image (inpainting)
- Upscaling and restyling
- Controlled generation from sketches or poses
Limits to know
- Text inside images and hands can still go wrong
- Copyright and likeness questions around training data
- Needs a capable GPU to run locally
