6.3 KiB
Pipelines
Pipelines provide a simple way to run state-of-the-art diffusion models in inference by bundling all of the necessary components (multiple independently-trained models, schedulers, and processors) into a single end-to-end class. Pipelines are flexible and they can be adapted to use different schedulers or even model components.
All pipelines are built from the base [DiffusionPipeline] class which provides basic functionality for loading, downloading, and saving all the components. Specific pipeline types (for example [StableDiffusionPipeline]) loaded with [~DiffusionPipeline.from_pretrained] are automatically detected and the pipeline components are loaded and passed to the __init__ function of the pipeline.
Warning
You shouldn't use the [
DiffusionPipeline] class for training. Individual components (for example, [UNet2DModel] and [UNet2DConditionModel]) of diffusion pipelines are usually trained individually, so we suggest directly working with them instead.
Pipelines do not offer any training functionality. You'll notice PyTorch's autograd is disabled by decorating the [
~DiffusionPipeline.__call__] method with atorch.no_graddecorator because pipelines should not be used for training. If you're interested in training, please take a look at the Training guides instead!
The table below lists all the pipelines currently available in 🤗 Diffusers and the tasks they support. Click on a pipeline to view its abstract and published paper.
| Pipeline | Tasks |
|---|---|
| aMUSEd | text2image |
| AnimateDiff | text2video |
| Attend-and-Excite | text2image |
| AudioLDM | text2audio |
| AudioLDM2 | text2audio |
| AuraFlow | text2image |
| BLIP Diffusion | text2image |
| Bria 3.2 | text2image |
| CogVideoX | text2video |
| Consistency Models | unconditional image generation |
| ControlNet | text2image, image2image, inpainting |
| ControlNet with Flux.1 | text2image |
| ControlNet with Hunyuan-DiT | text2image |
| ControlNet with Stable Diffusion 3 | text2image |
| ControlNet with Stable Diffusion XL | text2image |
| ControlNet-XS | text2image |
| ControlNet-XS with Stable Diffusion XL | text2image |
| Dance Diffusion | unconditional audio generation |
| DDIM | unconditional image generation |
| DDPM | unconditional image generation |
| DeepFloyd IF | text2image, image2image, inpainting, super-resolution |
| DiffEdit | inpainting |
| DiT | text2image |
| Flux | text2image |
| Hunyuan-DiT | text2image |
| I2VGen-XL | image2video |
| InstructPix2Pix | image editing |
| Kandinsky 2.1 | text2image, image2image, inpainting, interpolation |
| Kandinsky 2.2 | text2image, image2image, inpainting |
| Kandinsky 3 | text2image, image2image |
| Kolors | text2image |
| Latent Consistency Models | text2image |
| Latent Diffusion | text2image, super-resolution |
| Latte | text2image |
| LEDITS++ | image editing |
| Lumina-T2X | text2image |
| Marigold | depth-estimation, normals-estimation, intrinsic-decomposition |
| MultiDiffusion | text2image |
| MusicLDM | text2audio |
| PAG | text2image |
| Paint by Example | inpainting |
| PIA | image2video |
| PixArt-α | text2image |
| PixArt-Σ | text2image |
| Self-Attention Guidance | text2image |
| Semantic Guidance | text2image |
| Shap-E | text-to-3D, image-to-3D |
| Stable Audio | text2audio |
| Stable Cascade | text2image |
| Stable Diffusion | text2image, image2image, depth2image, inpainting, image variation, latent upscaler, super-resolution |
| Stable Diffusion XL | text2image, image2image, inpainting |
| Stable Diffusion XL Turbo | text2image, image2image, inpainting |
| Stable unCLIP | text2image, image variation |
| T2I-Adapter | text2image |
| Text2Video | text2video, video2video |
| Text2Video-Zero | text2video |
| unCLIP | text2image, image variation |
| UniDiffuser | text2image, image2text, image variation, text variation, unconditional image generation, unconditional audio generation |
| Value-guided planning | value guided sampling |
| Wuerstchen | text2image |
| VisualCloze | text2image, image2image, subject driven generation, inpainting, style transfer, image restoration, image editing, [depth,normal,edge,pose]2image, [depth,normal,edge,pose]-estimation, virtual try-on, image relighting |
DiffusionPipeline
autodoc DiffusionPipeline - all - call - device - to - components
autodoc pipelines.StableDiffusionMixin.enable_freeu
autodoc pipelines.StableDiffusionMixin.disable_freeu
PushToHubMixin
autodoc utils.PushToHubMixin
Callbacks
autodoc callbacks.PipelineCallback
autodoc callbacks.SDCFGCutoffCallback
autodoc callbacks.SDXLCFGCutoffCallback
autodoc callbacks.SDXLControlnetCFGCutoffCallback
autodoc callbacks.IPAdapterScaleCutoffCallback
autodoc callbacks.SD3CFGCutoffCallback