Commit Graph

771 Commits

Author SHA1 Message Date
Steven Liu
ceb53a65bd [docs] IP-Adapter (#6897)
* use cases

* first draft

* fix image links

* lcm-lora

* feedback

* review

* feedback

* feedback
2024-12-23 13:02:07 +05:30
Younes Belkada
484c256303 [PEFT / docs] Add a note about torch.compile (#6864)
* Update using_peft_for_inference.md

* add more explanation
2024-12-23 13:02:07 +05:30
Steven Liu
1b6feac509 [docs] Community pipelines (#6929)
fix
2024-12-23 13:02:07 +05:30
Sayak Paul
86e62ae4c5 change to 2024 in the license (#6902)
change to 2024
2024-12-23 13:02:07 +05:30
Félix Sanz
6d0fe4ea94 fix: typo in callback function name and property (#6834)
* fix: callback function name is incorrect

On this tutorial there is a function defined and then used inside `callback_on_step_end` argument, but the name was not correct (mismatch)

* fix: typo in num_timestep (correct is num_timesteps)

fixed property name
2024-12-23 13:02:06 +05:30
Dhruv Nair
d44a09475d Multiple small fixes to Video Pipeline docs (#6805)
* update

* update

* update

* Update src/diffusers/pipelines/i2vgen_xl/pipeline_i2vgen_xl.py

Co-authored-by: YiYi Xu <yixu310@gmail.com>

* update

* update

---------

Co-authored-by: YiYi Xu <yixu310@gmail.com>
2024-12-23 13:02:06 +05:30
Sayak Paul
804d50ec24 [Feat] add I2VGenXL for image-to-video generation (#6665)
---------

Co-authored-by: Dhruv Nair <dhruv.nair@gmail.com>
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
Co-authored-by: YiYi Xu <yixu310@gmail.com>
2024-12-23 13:02:06 +05:30
YiYi Xu
f1e105e664 [IP-Adapter] Support multiple IP-Adapters (#6573)
---------

Co-authored-by: yiyixuxu <yixu310@gmail,com>
Co-authored-by: Alvaro Somoza <somoza.alvaro@gmail.com>
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
2024-12-23 13:02:06 +05:30
Dhruv Nair
a31ebdc56c Add PIA Model/Pipeline (#6698)
* update

* update

* updaet

* add tests and docs

* clean up

* add to toctree

* fix copies

* pr review feedback

* fix copies

* fix tests

* update docs

* update

* update

* update docs

* update

* update

* update

* update
2024-12-23 13:02:06 +05:30
Sayak Paul
4485d34f2f add note about serialization (#6764) 2024-12-23 13:02:06 +05:30
xhedit
e0f29a6a85 Update lora.md with a more accurate description of rank (#6724)
* Update lora.md with a more accurate description of rank

* Update docs/source/en/training/lora.md

Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>

---------

Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
2024-12-23 13:02:06 +05:30
Sayak Paul
7bd61ef799 [Core] move transformer scripts to transformers modules (#6747)
* move transformer scripts to transformers modules

* move transformer model test

* move prior transformer test to  directory

* fix doc path

* correct doc path

* add: __init__.py
2024-12-23 13:02:06 +05:30
Aryan V S
6a11a89d76 [docs] AnimateDiff Video-to-Video (#6712)
* add animatediff vid2vid to docs

* Update docs/source/en/api/pipelines/animatediff.md

Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>

* apply suggestions from review

---------

Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
2024-12-23 13:02:06 +05:30
Steven Liu
663019577c [docs] UViT2D (#6643)
* uvit2d

* fix

* fix?

* add correct paper

* fix paths

* update abstract
2024-12-23 13:02:06 +05:30
Aryan V S
7d3f879501 AnimateDiff Video to Video (#6328)
* begin animatediff img2video and video2video

* revert animatediff to original implementation

* add img2video as pipeline

* update

* add vid2vid pipeline

* update imports

* update

* remove copied from line for check_inputs

* update

* update examples

* add multi-batch support

* fix __init__.py files

* move img2vid to community

* update community readme and examples

* fix

* make fix-copies

* add vid2vid batch params

* apply suggestions from review

Co-Authored-By: Dhruv Nair <dhruv.nair@gmail.com>

* add test for animatediff vid2vid

* torch.stack -> torch.cat

Co-Authored-By: Dhruv Nair <dhruv.nair@gmail.com>

* make style

* docs for vid2vid

* update

* fix prepare_latents

* fix docs

* remove img2vid

* update README to :main

* remove slow test

* refactor pipeline output

* update docs

* update docs

* merge community readme from :main

* final fix i promise

* add support for url in animatediff example

* update example

* update callbacks to latest implementation

* Update src/diffusers/pipelines/animatediff/pipeline_animatediff_video2video.py

Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>

* Update src/diffusers/pipelines/animatediff/pipeline_animatediff_video2video.py

Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>

* fix merge

* Apply suggestions from code review

* remove callback and callback_steps as suggested in review

* Update tests/pipelines/animatediff/test_animatediff_video2video.py

Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>

* fix import error caused due to unet refactor in #6630

* fix numpy import error after tensor2vid refactor in #6626

* make fix-copies

* fix numpy error

* fix progress bar test

---------

Co-authored-by: Dhruv Nair <dhruv.nair@gmail.com>
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
2024-12-23 13:02:06 +05:30
Dhruv Nair
20ecdf6fc3 [Refactor] Update from single file (#6428)
* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update'

* update

* update

* update

* update

* update

* update

* up

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* up

* update

* update

* update

* update

* update'

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* clean

* update

* update

* clean up

* clean up

* update

* clean

* clean

* update

* updaet

* clean up

* fix docs

* update

* update

* Revert "update"

This reverts commit dbfb8f1ea9.

* update

* update

* update

* update

* fix controlnet

* fix scheduler

* fix controlnet tests
2024-12-23 13:02:06 +05:30
Sayak Paul
48ddd0a803 [Big refactor] move unets to unets module 🦋 (#6630)
* move unets to  module 🦋

* parameterize unet-level import.

* fix flax unet2dcondition model import

* models __init__

* mildly depcrecating models.unet_2d_blocks in favor of models.unets.unet_2d_blocks.

* noqa

* correct depcrecation behaviour

* inherit from the actual classes.

* Empty-Commit

* backwards compatibility for unet_2d.py

* backward compatibility for unet_2d_condition

* bc for unet_1d

* bc for unet_1d_blocks
2024-12-23 13:02:06 +05:30
Steven Liu
090a9eb624 [docs] Fix missing API function (#6604)
fix?
2024-12-23 13:02:06 +05:30
Aryan V S
f3e47f4c70 FreeInit (#6315)
* freeinit

* update freeinit implementation based on review

Co-Authored-By: Dhruv Nair <dhruv.nair@gmail.com>

* fix

* another fix

* refactor

* fix timesteps missing bug

* apply suggestions from review

Co-Authored-By: Dhruv Nair <dhruv.nair@gmail.com>

* add test for freeinit

* apply suggestions from review

Co-Authored-By: Dhruv Nair <dhruv.nair@gmail.com>

* refactor

* fix test

* fix tensor not on same device

* update

* remove return_intermediate_results

* fix broken freeinit test

* update animatediff docs

---------

Co-authored-by: Dhruv Nair <dhruv.nair@gmail.com>
2024-12-23 13:02:06 +05:30
Bhavay Malhotra
1ac35a457e Update installation.md (#6438)
* Update installation.md

* Update installation.md

* Update installation.md
2024-12-23 13:02:06 +05:30
Sayak Paul
9017578d88 remove omegaconf from the residues 👋 (#6600)
remove omegaconf 👋
2024-12-23 13:02:06 +05:30
JuanCarlosPi
c9ffa6cd90 Change in ip-adapter docs. CLIPVisionModelWithProjection should be im… (#6597)
Change in ip-adapter docs. CLIPVisionModelWithProjection should be imported from transformers, not diffusers
2024-12-23 13:02:06 +05:30
Patrick von Platen
3654809062 [Docs] Fix controlnet indent (#6578) 2024-12-23 13:02:05 +05:30
Steven Liu
05322fb98d [docs] mask_blur and padding_mask_crop (#6498)
new inpaint features
2024-12-23 13:02:05 +05:30
Steven Liu
1a0673574e [docs] Callbacks (#6471)
edits
2024-12-23 13:02:05 +05:30
YiYi Xu
b1b6a90be3 add StableDiffusionXLKDiffusionPipeline (#6447)
---------

Co-authored-by: yiyixuxu <yixu310@gmail,com>
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
2024-12-23 13:02:05 +05:30
Steven Liu
db484b5494 [docs] Stable video diffusion (#6472)
svd
2024-12-23 13:02:05 +05:30
Steven Liu
699b33da95 [docs] PEFT adapter API (#6499)
follow up
2024-12-23 13:02:05 +05:30
Steven Liu
9d2caa3f74 [docs] Fast diffusion (#6470)
* edits

* fix

* feedback
2024-12-23 13:02:05 +05:30
Sayak Paul
a8129030a8 minor changes to the SVD doc (#6466)
minor changes
2024-12-23 13:02:05 +05:30
Sayak Paul
38c3bcd840 [Core] introduce PeftAdapterMixin module. (#6416)
* introduce integrations module.

* remove duplicate methods.

* better imports.

* move to loaders.py

* remove peftadaptermixin from modelmixin.

* add: peftadaptermixin selectively.

* add: entry to _toctree

* Empty-Commit
2024-12-23 13:02:05 +05:30
Liang Hou
799d294f42 Fix amused paper link (#6462) 2024-12-23 13:02:05 +05:30
Horseee
4a0d0cf2ed [Doc] Add DeepCache in section optimization/General optimizations (#6390)
* add documentation for DeepCache

* fix typo

* add wandb url for DeepCache

* fix some typos

* add item in _toctree.yml

* update formats for arguments

* Update deepcache.md

* Update docs/source/en/optimization/deepcache.md

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>

* add StableDiffusionXLPipeline in doc

* Separate SDPipeline and SDXLPipeline

* Add the paper link of ablation experiments for hyper-parameters

* Apply suggestions from code review

Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>

---------

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
2024-12-23 13:02:05 +05:30
Steven Liu
baf9199ac5 [docs] Fix local links (#6440)
fix local links
2024-12-23 13:02:05 +05:30
Sayak Paul
6fc757690b add: amused paper link. (#6453) 2024-12-23 13:02:05 +05:30
YiYi Xu
e0faf4b00f [doc] update inpaint doc to use apply_overlay (#6364)
add doc

Co-authored-by: yiyixuxu <yixu310@gmail,com>
2024-12-23 13:02:05 +05:30
Sayak Paul
06d6b30caf add: CUDA graph details. (#6408) 2024-12-23 13:02:05 +05:30
Sayak Paul
aeec2f90f4 [Docs] add note on fp16 in fast diffusion (#6380)
add note on fp16
2024-12-23 13:02:05 +05:30
Sayak Paul
140bdfde06 [docs] add details concerning diffusers-specific bits. (#6375)
add details concerning diffusers-specific bits.
2024-12-23 13:02:05 +05:30
Dhruv Nair
47308a91f6 Move ControlNetXS into Community Folder (#6316)
* update

* update

* update

* update

* update

* make style

* remove docs

* update

* move to research folder.

* fix-copies

* remove _toctree entry.

---------

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
2024-12-23 13:02:05 +05:30
Will Berman
1de21a5996 amused update links to new repo (#6344)
* amused update links to new repo

* lint
2024-12-23 13:02:05 +05:30
Sayak Paul
7570fcef40 [Docs] fix: video rendering on svd. (#6330)
fix: video rendering on svd.
2024-12-23 13:02:05 +05:30
Will Berman
98f2ae0011 amused other pipelines docs (#6343)
other pipelines
2024-12-23 13:02:05 +05:30
Dhruv Nair
f959a96fe4 Update Animatediff docs (#6341)
* update

* update

* update
2024-12-23 13:02:05 +05:30
Dhruv Nair
d310bf3589 Interruptable Pipelines (#5867)
* add interruptable pipelines

* add tests

* updatemsmq

* add interrupt property

* make fix copies

* Revert "make fix copies"

This reverts commit 914b35332b.

* add docs

* add tutorial

* Update docs/source/en/tutorials/interrupting_diffusion_process.md

Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>

* Update docs/source/en/tutorials/interrupting_diffusion_process.md

Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>

* update

* fix quality issues

* fix

* update

---------

Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
2024-12-23 13:02:05 +05:30
Sayak Paul
14ab0ead5e [Diffusion fast] add doc for diffusion fast (#6311)
* add doc for diffusion fast

* add entry to _toctree

* Apply suggestions from code review

* fix titlew

* fix: title entry

* add note about fuse_qkv_projections
2024-12-23 13:02:05 +05:30
Younes Belkada
c07374f725 [Peft / Lora] Add adapter_names in fuse_lora (#5823)
* add adapter_name in fuse

* add tesrt

* up

* fix CI

* adapt from suggestion

* Update src/diffusers/utils/testing_utils.py

Co-authored-by: Benjamin Bossan <BenjaminBossan@users.noreply.github.com>

* change to `require_peft_version_greater`

* change variable names in test

* Update src/diffusers/loaders/lora.py

Co-authored-by: Benjamin Bossan <BenjaminBossan@users.noreply.github.com>

* break into 2 lines

* final comments

---------

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
Co-authored-by: Benjamin Bossan <BenjaminBossan@users.noreply.github.com>
2024-12-23 13:02:05 +05:30
Sayak Paul
7727fa6926 fix: t2i apdater paper link (#6314) 2024-12-23 13:02:05 +05:30
Will Berman
8d0bf4f3da open muse (#5437)
amused

rename

Update docs/source/en/api/pipelines/amused.md

Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>

AdaLayerNormContinuous default values

custom micro conditioning

micro conditioning docs

put lookup from codebook in constructor

fix conversion script

remove manual fused flash attn kernel

add training script

temp remove training script

add dummy gradient checkpointing func

clarify temperatures is an instance variable by setting it

remove additional SkipFF block args

hardcode norm args

rename tests folder

fix paths and samples

fix tests

add training script

training readme

lora saving and loading

non-lora saving/loading

some readme fixes

guards

Update docs/source/en/api/pipelines/amused.md

Co-authored-by: Suraj Patil <surajp815@gmail.com>

Update examples/amused/README.md

Co-authored-by: Suraj Patil <surajp815@gmail.com>

Update examples/amused/train_amused.py

Co-authored-by: Suraj Patil <surajp815@gmail.com>

vae upcasting

add fp16 integration tests

use tuple for micro cond

copyrights

remove casts

delegate to torch.nn.LayerNorm

move temperature to pipeline call

upsampling/downsampling changes
2024-12-23 13:02:04 +05:30
Sayak Paul
b7ff3022ee [Refactor] move ldm3d out of stable_diffusion. (#6263)
ldm3d.
2024-12-23 13:02:04 +05:30