Aryan
f33b233789
[bug] remove unreachable norm_type=ada_norm_continuous from norm3 initialization conditions ( #9006 )
...
remove ada_norm_continuous from norm3 list
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
2024-12-23 13:02:15 +05:30
Sayak Paul
1616a6372e
[Core] add QKV fusion to AuraFlow and PixArt Sigma ( #8952 )
...
* add fusion support to pixart
* add to auraflow.
* add tests
* apply review feedback.
* add back args and kwargs
* style
2024-12-23 13:02:15 +05:30
Tolga Cangöz
51f45da25f
Update CLIPFeatureExtractor to CLIPImageProcessor and DPTFeatureExtractor to DPTImageProcessor ( #9002 )
...
* fix: update `CLIPFeatureExtractor` to `CLIPImageProcessor` in codebase
* `make style && make quality`
* Update `DPTFeatureExtractor` to `DPTImageProcessor` in codebase
* `make style`
---------
Co-authored-by: Aryan <aryan@huggingface.co >
2024-12-23 13:02:15 +05:30
omahs
49544cc1fb
Fix typos ( #9077 )
...
* fix typo
2024-12-23 13:02:15 +05:30
YiYi Xu
627fd46ab8
add sentencepiece as a soft dependency ( #9065 )
...
* add sentencepiece as soft dependency for kolors
* up
---------
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
2024-12-23 13:02:15 +05:30
Sayak Paul
8881fc9872
[Docs] add stable cascade unet doc. ( #9066 )
...
* add stable cascade unet doc.
* fix path
2024-12-23 13:02:15 +05:30
Aryan
9dbffc8c60
PAG variant for HunyuanDiT, PAG refactor ( #8936 )
...
* copy hunyuandit pipeline
* pag variant of hunyuan dit
* add tests
* update docs
* make style
* make fix-copies
* Update src/diffusers/pipelines/pag/pag_utils.py
* remove incorrect copied from
* remove pag hunyuan attn procs to resolve conflicts
* add pag attn procs again
* new implementation for pag_utils
* revert pag changes
* add pag refactor back; update pixart sigma
* update pixart pag tests
* apply suggestions from review
Co-Authored-By: yixu310@gmail.com
* make style
* update docs, fix tests
* fix tests
* fix test_components_function since list not accepted as valid __init__ param
* apply patch to fix broken tests
Co-Authored-By: Sayak Paul <spsayakpaul@gmail.com >
* make style
* fix hunyuan tests
---------
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
2024-12-23 13:02:15 +05:30
Vinh H. Pham
fa55429b04
[Tests] Improve transformers model test suite coverage - Latte ( #8919 )
...
* add LatteTransformer3DModel model test
* change patch_size to 1
* reduce req len
* reduce channel dims
* increase num_layers
* reduce dims further
* run make style
---------
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
Co-authored-by: Aryan <aryan@huggingface.co >
2024-12-23 13:02:15 +05:30
Sayak Paul
499b7d6dde
[FLUX] support LoRA ( #9057 )
...
* feat: lora support for Flux.
add tests
fix imports
major fixes.
* fix
fixes
final fixes?
* fix
* remove is_peft_available.
2024-12-23 13:02:15 +05:30
Aryan
44a4886771
[refactor] apply qk norm in attention processors ( #9071 )
...
* apply qk norm in attention processors
* revert attention processor
* qk-norm in only attention proc 2.0 and fused variant
2024-12-23 13:02:15 +05:30
psychedelicious
01829c699a
type get_attention_scores as optional in get_attention_scores ( #9075 )
...
`None` is valid for `get_attention_scores`, should be typed as such
2024-12-23 13:02:15 +05:30
asfiyab-nvidia
fce5debd8c
Update TensorRT txt2img and inpaint community pipelines ( #9037 )
...
* Update TensorRT txt2img and inpaint community pipelines
Signed-off-by: Asfiya Baig <asfiyab@nvidia.com >
* update tensorrt install instructions
Signed-off-by: Asfiya Baig <asfiyab@nvidia.com >
---------
Signed-off-by: Asfiya Baig <asfiyab@nvidia.com >
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
2024-12-23 13:02:15 +05:30
Sayak Paul
0db81141b9
[Flux] minor documentation fixes for flux. ( #9048 )
...
* minor documentation fixes for flux.
* clipskip
* add gist
2024-12-23 13:02:15 +05:30
Philip Rideout
9c02c40a13
Fix grammar mistake. ( #9072 )
2024-12-23 13:02:15 +05:30
Aryan
6e5b374630
[refactor] create modeling blocks specific to AnimateDiff ( #8979 )
...
* animatediff specific transformer model
* make style
* make fix-copies
* move blocks to unet motion model
* make style
* remove dummy object
* fix incorrectly passed param causing test failures
* rename model and output class
* fix sparsectrl imports
* remove todo comments
* remove temporal double self attn param from controlnet sparsectrl
* add deprecated versions of blocks
* apply suggestions from review
* update
---------
Co-authored-by: Dhruv Nair <dhruv.nair@gmail.com >
2024-12-23 13:02:15 +05:30
Tolga Cangöz
c6ac793955
Errata: Fix typos & \s+$ ( #9008 )
...
* Fix typos
* chore: Fix typos
* chore: Update README.md for promptdiffusion example
* Trim trailing white spaces
* Fix a typo
* update number
* chore: update number
* Trim trailing white space
* Update README.md
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
* Update README.md
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
---------
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
2024-12-23 13:02:14 +05:30
Frank (Haofan) Wang
9769fae865
Update transformer_flux.py ( #9060 )
2024-12-23 13:02:14 +05:30
Dhruv Nair
a615e524e5
Fix Nightly Deps ( #9036 )
...
update
2024-12-23 13:02:14 +05:30
Sayak Paul
fdce85ccf1
[Flux] allow tests to run ( #9050 )
...
* fix tests
* fix
* float64 skip
* remove sample_size.
* remove
* remove more
* default_sample_size.
* credit black forest for flux model.
* skip
* fix: tests
* remove OriginalModelMixin
* add transformer model test
* add: transformer model tests
2024-12-23 13:02:14 +05:30
Sayak Paul
c8a236ba5c
[Core] Add PAG support for PixArtSigma ( #8921 )
...
* feat: add pixart sigma pag.
* inits.
* fixes
* fix
* remove print.
* copy paste methods to the pixart pag mixin
* fix-copies
* add documentation.
* add tests.
* remove correction file.
* remove pag_applied_layers
* empty
2024-12-23 13:02:14 +05:30
Sayak Paul
7739beb740
Flux pipeline ( #9043 )
...
add flux!
Signed-off-by: Adrien <adrien@huggingface.co >
Co-authored-by: Adrien <adrien.69740@gmail.com >
Co-authored-by: Anatoly Belikov <abelikov@singularitynet.io >
Co-authored-by: Dhruv Nair <dhruv.nair@gmail.com >
Co-authored-by: yiyixuxu <yixu310@gmail.com >
2024-12-23 13:02:14 +05:30
Aryan
6f90bc1a63
[docs] fix pia example ( #9015 )
...
fix pia example docstring
2024-12-23 13:02:14 +05:30
YiYi Xu
ceeaf1d469
fix load sharded checkpoint from a subfolder (local path) ( #8913 )
...
fix
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
2024-12-23 13:02:14 +05:30
Dhruv Nair
232a967613
Updates deps for pipeline test fetcher ( #9033 )
...
update
2024-12-23 13:02:14 +05:30
Aryan
e28e5373f9
PAG variant for AnimateDiff ( #8789 )
...
* add animatediff pag pipeline
* remove unnecessary print
* make fix-copies
* fix ip-adapter bug
* update docs
* add fast tests and fix bugs
* update
* update
* address review comments
* update ip adapter single test expected slice
* implement test_from_pipe_consistent_config; fix expected slice values
* LoraLoaderMixin->StableDiffusionLoraLoaderMixin; add latest freeinit test
2024-12-23 13:02:14 +05:30
Yoach Lacombe
8c154daddd
Fix Stable Audio repository id ( #9016 )
...
Fix Stable Audio repo id
2024-12-23 13:02:14 +05:30
Aryan
cf513e4205
[core] Move community AnimateDiff ControlNet to core ( #8972 )
...
* add animatediff controlnet to core
* make style; remove unused method
* fix copied from comment
* add tests
* changes to make tests work
* add utility function to load videos
* update docs
* update pipeline example
* make style
* update docs with example
* address review comments
* add latest freeinit test from #8969
* LoraLoaderMixin -> StableDiffusionLoraLoaderMixin
* fix docs
* Update src/diffusers/utils/loading_utils.py
Co-authored-by: Dhruv Nair <dhruv.nair@gmail.com >
* fix: variable out of scope
---------
Co-authored-by: Dhruv Nair <dhruv.nair@gmail.com >
2024-12-23 13:02:14 +05:30
Yoach Lacombe
030a134311
Stable Audio integration ( #8716 )
...
* WIP modeling code and pipeline
* add custom attention processor + custom activation + add to init
* correct ProjectionModel forward
* add stable audio to __initèè
* add autoencoder and update pipeline and modeling code
* add half Rope
* add partial rotary v2
* add temporary modfis to scheduler
* add EDM DPM Solver
* remove TODOs
* clean GLU
* remove att.group_norm to attn processor
* revert back src/diffusers/schedulers/scheduling_dpmsolver_multistep.py
* refactor GLU -> SwiGLU
* remove redundant args
* add channel multiples in autoencoder docstrings
* changes in docsrtings and copyright headers
* clean pipeline
* further cleaning
* remove peft and lora and fromoriginalmodel
* Delete src/diffusers/pipelines/stable_audio/diffusers.code-workspace
* make style
* dummy models
* fix copied from
* add fast oobleck tests
* add brownian tree
* oobleck autoencoder slow tests
* remove TODO
* fast stable audio pipeline tests
* add slow tests
* make style
* add first version of docs
* wrap is_torchsde_available to the scheduler
* fix slow test
* test with input waveform
* add input waveform
* remove some todos
* create stableaudio gaussian projection + make style
* add pipeline to toctree
* fix copied from
* make quality
* refactor timestep_features->time_proj
* refactor joint_attention_kwargs->cross_attention_kwargs
* remove forward_chunk
* move StableAudioDitModel to transformers folder
* correct convert + remove partial rotary embed
* apply suggestions from yiyixuxu -> removing attn.kv_heads
* remove temb
* remove cross_attention_kwargs
* further removal of cross_attention_kwargs
* remove text encoder autocast to fp16
* continue removing autocast
* make style
* refactor how text and audio are embedded
* add paper
* update example code
* make style
* unify projection model forward + fix device placement
* make style
* remove fuse qkv
* apply suggestions from review
* Update src/diffusers/pipelines/stable_audio/pipeline_stable_audio.py
Co-authored-by: YiYi Xu <yixu310@gmail.com >
* make style
* smaller models in fast tests
* pass sequential offloading fast tests
* add docs for vae and autoencoder
* make style and update example
* remove useless import
* add cosine scheduler
* dummy classes
* cosine scheduler docs
* better description of scheduler
---------
Co-authored-by: YiYi Xu <yixu310@gmail.com >
2024-12-23 13:02:14 +05:30
Sayak Paul
1a903d8019
[LoRA] fix: animate diff lora stuff. ( #8995 )
...
* fix: animate diff lora stuff.
* fix scaling function for UNetMotionModel
* emoty
2024-12-23 13:02:14 +05:30
Anatoly Belikov
c7452308f5
handle lora scale and clip skip in lpw sd and sdxl community pipelines ( #8988 )
...
* handle lora scale and clip skip in lpw sd and sdxl
* use StableDiffusionLoraLoaderMixin
* use StableDiffusionXLLoraLoaderMixin
* style
---------
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
2024-12-23 13:02:14 +05:30
Sayak Paul
3566f4b18a
[Docs] credit where it's due for Lumina and Latte. ( #9000 )
...
credit where it's due for Lumina and Latte.
2024-12-23 13:02:14 +05:30
Adrien
f87ef1d061
[CI] Update runner configuration for setup and nightly tests ( #9005 )
...
* [CI] Update runner configuration for setup and nightly tests
Signed-off-by: Adrien <adrien@huggingface.co >
* fix group
Signed-off-by: Adrien <adrien@huggingface.co >
* update for t4
Signed-off-by: Adrien <adrien@huggingface.co >
---------
Signed-off-by: Adrien <adrien@huggingface.co >
2024-12-23 13:02:14 +05:30
Álvaro Somoza
edddf3d417
[Kolors] Add IP Adapter ( #8901 )
...
* initial draft
* apply suggestions
* fix failing test
* added ipa to img2img
* add docs
* apply suggestions
2024-12-23 13:02:14 +05:30
Aryan
a9de5cf59a
remove unused code from pag attn procs ( #8928 )
2024-12-23 13:02:14 +05:30
Aryan
b7ddd2bb99
[core] AnimateDiff SparseCtrl ( #8897 )
...
* initial sparse control model draft
* remove unnecessary implementation
* copy animatediff pipeline
* remove deprecated callbacks
* update
* update pipeline implementation progress
* make style
* make fix-copies
* update progress
* add partially working pipeline
* remove debug prints
* add model docs
* dummy objects
* improve motion lora conversion script
* fix bugs
* update docstrings
* remove unnecessary model params; docs
* address review comment
* add copied from to zero_module
* copy animatediff test
* add fast tests
* update docs
* update
* update pipeline docs
* fix expected slice values
* fix license
* remove get_down_block usage
* remove temporal_double_self_attention from get_down_block
* update
* update docs with org and documentation images
* make from_unet work in sparsecontrolnetmodel
* add latest freeinit test from #8969
* make fix-copies
* LoraLoaderMixin -> StableDiffsuionLoraLoaderMixin
2024-12-23 13:02:14 +05:30
Aryan
5e532e58cf
[fix] FreeInit step index out of bounds ( #8969 )
...
* fix step index out of bounds
* add test for free_init with different schedulers
* add test to vid2vid and pia
2024-12-23 13:02:14 +05:30
Dhruv Nair
ca299f0430
[CI] Nightly Test Runner explicitly set runner for Setup Pipeline Matrix ( #8986 )
...
* update
* update
* update
2024-12-23 13:02:14 +05:30
Dhruv Nair
e5113511cd
[CI] Fix parallelism in nightly tests ( #8983 )
...
update
2024-12-23 13:02:14 +05:30
RandomGamingDev
13ef7e1b98
Added accelerator based gradient accumulation for basic_example ( #8966 )
...
added accelerator based gradient accumulation for basic_example
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
2024-12-23 13:02:14 +05:30
Sayak Paul
6d11129c5a
[Chore] add LoraLoaderMixin to the inits ( #8981 )
...
* introduce to promote reusability.
* up
* add more tests
* up
* remove comments.
* fix fuse_nan test
* clarify the scope of fuse_lora and unfuse_lora
* remove space
* rewrite fuse_lora a bit.
* feedback
* copy over load_lora_into_text_encoder.
* address dhruv's feedback.
* fix-copies
* fix issubclass.
* num_fused_loras
* fix
* fix
* remove mapping
* up
* fix
* style
* fix-copies
* change to SD3TransformerLoRALoadersMixin
* Apply suggestions from code review
Co-authored-by: Dhruv Nair <dhruv.nair@gmail.com >
* up
* handle wuerstchen
* up
* move lora to lora_pipeline.py
* up
* fix-copies
* fix documentation.
* comment set_adapters().
* fix-copies
* fix set_adapters() at the model level.
* fix?
* fix
* loraloadermixin.
---------
Co-authored-by: Dhruv Nair <dhruv.nair@gmail.com >
2024-12-23 13:02:14 +05:30
Sayak Paul
999e8c4496
[Chore] remove all is from auraflow. ( #8980 )
...
remove all is from auraflow.
2024-12-23 13:02:14 +05:30
efwfe
5473b3d475
fix guidance_scale value not equal to the value in comments ( #8941 )
...
fix guidance_scale value not equal with the value in comments
2024-12-23 13:02:14 +05:30
YiYi Xu
a754d9071e
Revert "[LoRA] introduce LoraBaseMixin to promote reusability." ( #8976 )
...
Revert "[LoRA] introduce LoraBaseMixin to promote reusability. (#8774 )"
This reverts commit 527430d0a4
2024-12-23 13:02:14 +05:30
mazharosama
7f74c09107
Enable CivitAI SDXL Inpainting Models Conversion ( #8795 )
...
modify in_channels in network_config params
Co-authored-by: Dhruv Nair <dhruv.nair@gmail.com >
2024-12-23 13:02:14 +05:30
asfiyab-nvidia
d1e1676b9d
Update TensorRT img2img community pipeline ( #8899 )
...
* Update TensorRT img2img pipeline
Signed-off-by: Asfiya Baig <asfiyab@nvidia.com >
* Update TensorRT version installed
Signed-off-by: Asfiya Baig <asfiyab@nvidia.com >
* make style and quality
Signed-off-by: Asfiya Baig <asfiyab@nvidia.com >
* Update examples/community/stable_diffusion_tensorrt_img2img.py
Co-authored-by: Tolga Cangöz <46008593+tolgacangoz@users.noreply.github.com >
* Update examples/community/README.md
Co-authored-by: Tolga Cangöz <46008593+tolgacangoz@users.noreply.github.com >
* Apply style and quality using ruff 0.1.5
Signed-off-by: Asfiya Baig <asfiyab@nvidia.com >
---------
Signed-off-by: Asfiya Baig <asfiyab@nvidia.com >
Co-authored-by: Tolga Cangöz <46008593+tolgacangoz@users.noreply.github.com >
Co-authored-by: Dhruv Nair <dhruv.nair@gmail.com >
2024-12-23 13:02:14 +05:30
Sayak Paul
82b37a4cc3
[LoRA] introduce LoraBaseMixin to promote reusability. ( #8774 )
...
* introduce to promote reusability.
* up
* add more tests
* up
* remove comments.
* fix fuse_nan test
* clarify the scope of fuse_lora and unfuse_lora
* remove space
* rewrite fuse_lora a bit.
* feedback
* copy over load_lora_into_text_encoder.
* address dhruv's feedback.
* fix-copies
* fix issubclass.
* num_fused_loras
* fix
* fix
* remove mapping
* up
* fix
* style
* fix-copies
* change to SD3TransformerLoRALoadersMixin
* Apply suggestions from code review
Co-authored-by: Dhruv Nair <dhruv.nair@gmail.com >
* up
* handle wuerstchen
* up
* move lora to lora_pipeline.py
* up
* fix-copies
* fix documentation.
* comment set_adapters().
* fix-copies
* fix set_adapters() at the model level.
* fix?
* fix
---------
Co-authored-by: Dhruv Nair <dhruv.nair@gmail.com >
2024-12-23 13:02:14 +05:30
Aryan
41f79e9576
[tests] speed up animatediff tests ( #8846 )
...
* speed up animatediff tests
* fix pia test_ip_adapter_single
* fix tests/pipelines/pia/test_pia.py::PIAPipelineFastTests::test_dict_tuple_outputs_equivalent
* update
* fix ip adapter tests
* skip test_from_pipe_consistent_config tests
* fix prompt_embeds test
* update test_from_pipe_consistent_config tests
* fix expected_slice values
* remove temporal_norm_num_groups from UpBlockMotion
---------
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
Co-authored-by: Dhruv Nair <dhruv.nair@gmail.com >
2024-12-23 13:02:14 +05:30
Dhruv Nair
e42c333819
[CI] Slow Test Updates ( #8870 )
...
* update
* update
* update
2024-12-23 13:02:14 +05:30
Sayak Paul
015019ab7d
[Tests] fix slices of 26 tests (first half) ( #8959 )
...
* check for assertions.
* update with correct slices.
* okay
* style
* get it ready
* update
* update
* update
---------
Co-authored-by: Dhruv Nair <dhruv.nair@gmail.com >
2024-12-23 13:02:14 +05:30
Sanchit Gandhi
bc70d92317
[AudioLDM2] Fix cache pos for GPT-2 generation ( #8964 )
...
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
2024-12-23 13:02:14 +05:30
