LLM/diffusers
1
0
Fork 0
mirror of https://github.com/huggingface/diffusers.git synced 2026-02-14 14:55:26 +08:00
Code Issues Packages Projects Releases Wiki Activity
4,913 Commits 743 Branches 94 Tags
4c5082ca685d1dd0a24907890da170e258e5cd01
Commit Graph

4913 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Sayak Paul
32f85ed014 chore: add a cleaning utility to be useful during training. (#9240) 2024-12-23 13:02:16 +05:30
Aryan
c636d53456 [refactor] move positional embeddings to patch embed layer for CogVideoX (#9263) 
* remove frame limit in cogvideox

* remove debug prints

* Update src/diffusers/models/transformers/cogvideox_transformer_3d.py

* revert pipeline; remove frame limitation

* revert transformer changes

* address review comments

* add error message

* apply suggestions from review
 2024-12-23 13:02:16 +05:30
Dhruv Nair
f16a47f8b0 [CI] Add option to dispatch Fast GPU tests on main (#9355) 
update
 2024-12-23 13:02:16 +05:30
Aryan
7dc5fd05f2 [tests] remove/speedup some low signal tests (#9285) 
* remove 2 shapes from SDFunctionTesterMixin::test_vae_tiling

* combine freeu enable/disable test to reduce many inference runs

* remove low signal unet test for signature

* remove low signal embeddings test

* remove low signal progress bar test from PipelineTesterMixin

* combine ip-adapter single and multi tests to save many inferences

* fix broken tests

* Update tests/pipelines/test_pipelines_common.py

* Update tests/pipelines/test_pipelines_common.py

* add progress bar tests
 2024-12-23 13:02:16 +05:30
Dhruv Nair
0354a45f47 [CI] More Fast GPU Test Fixes (#9346) 
* update

* update

* update

* update
 2024-12-23 13:02:16 +05:30
Vishnu V Jaddipal
12d1663e4f Xlabs lora fix (#9348) 
* Fix ```from_single_file``` for xl_inpaint

* Add basic flux inpaint pipeline

* style, quality, stray print

* Fix stray changes

* Add inpainting model support

* Change lora conversion for xlabs

* Fix stray changes

* Apply suggestions from code review

* style

---------

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
 2024-12-23 13:02:16 +05:30
Dhruv Nair
d171e6a5ea [CI] More fixes for Fast GPU Tests on main (#9300) 
update
 2024-12-23 13:02:16 +05:30
Aryan
91168b5436 [core] Support VideoToVideo with CogVideoX (#9333) 
* add vid2vid pipeline for cogvideox

* make fix-copies

* update docs

* fake context parallel cache, vae encode tiling

* add test for cog vid2vid

* use video link from HF docs repo

* add copied from comments; correctly rename test class
 2024-12-23 13:02:16 +05:30
Aryan
c466106eb8 [core] CogVideoX memory optimizations in VAE encode (#9340) 
fake context parallel cache, vae encode tiling

(cherry picked from commit bf890bca0e)
 2024-12-23 13:02:16 +05:30
YiYi Xu
4d901f9029 update runway repo for single_file (#9323) 
update to a place holder
 2024-12-23 13:02:15 +05:30
Aryan
ae69462e0e [docs] Add a note on torchao/quanto benchmarks for CogVideoX and memory-efficient inference (#9296) 
* add a note on torchao/quanto benchmarks and memory-efficient inference

* apply suggestions from review

* update

* Update docs/source/en/api/pipelines/cogvideox.md

Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>

* Update docs/source/en/api/pipelines/cogvideox.md

Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>

* add note on enable sequential cpu offload

---------

Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
 2024-12-23 13:02:15 +05:30
Dhruv Nair
b6cd4a4337 [CI] Update Hub Token on nightly tests (#9318) 
update
 2024-12-23 13:02:15 +05:30
YiYi Xu
ae8d9bb77a refactor rotary embedding 3: so it is not on cpu (#9307) 
change get_1d_rotary to accept pos as torch tensors
 2024-12-23 13:02:15 +05:30
YiYi Xu
17b838f5ae rotary embedding refactor 2: update comments, fix dtype for use_real=False (#9312) 
fix notes and dtype
 2024-12-23 13:02:15 +05:30
Anand Kumar
fdb1759b48 [train_custom_diffusion.py] Fix the LR schedulers when num_train_epochs is passed in a distributed training env (#9308) 
* Update train_custom_diffusion.py to fix the LR schedulers for `num_train_epochs`

* Fix saving text embeddings during safe serialization

* Fixed formatting
 2024-12-23 13:02:15 +05:30
Sayak Paul
348578e4bc [LoRA] support kohya and xlabs loras for flux. (#9295) 
* support kohya lora in flux.

* format

* support xlabs

* diffusion_model prefix.

* Apply suggestions from code review

Co-authored-by: apolinário <joaopaulo.passos@gmail.com>

* empty commit.

Co-authored-by: Leommm-byte <leom20031@gmail.com>

---------

Co-authored-by: apolinário <joaopaulo.passos@gmail.com>
Co-authored-by: Leommm-byte <leom20031@gmail.com>
 2024-12-23 13:02:15 +05:30
apolinário
7cfc8bd488 Change default for guidance_scalein FLUX (#9305) 
To match the original code, 7.0 is too high
 2024-12-23 13:02:15 +05:30
Aryan
34b6db7d24 AnimateDiff prompt travel (#9231) 
* update

* implement prompt interpolation

* make style

* resnet memory optimizations

* more memory optimizations; todo: refactor

* update

* update animatediff controlnet with latest changes

* refactor chunked inference changes

* remove print statements

* undo memory optimization changes

* update docstrings

* fix tests

* fix pia tests

* apply suggestions from review

* add tests

* update comment
 2024-12-23 13:02:15 +05:30
Frank (Haofan) Wang
b71158d2ec Update __init__.py (#9286) 2024-12-23 13:02:15 +05:30
Dhruv Nair
53721b9758 Fix Flux CLIP prompt embeds repeat for num_images_per_prompt > 1 (#9280) 
update
 2024-12-23 13:02:15 +05:30
Dhruv Nair
76c402e233 [CI] Update Release Tests (#9274) 
* update

* update
 2024-12-23 13:02:15 +05:30
Dhruv Nair
dc014c46e3 Fix Freenoise for AnimateDiff V3 checkpoint. (#9288) 
update
 2024-12-23 13:02:15 +05:30
Marçal Comajoan Cara
433dce672b Replace transformers.deepspeed with transformers.integrations.deepspeed (#9281) 
to avoid "FutureWarning: transformers.deepspeed module is deprecated and will be removed in a future version. Please import deepspeed modules directly from transformers.integrations"

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
 2024-12-23 13:02:15 +05:30
Steven Liu
4fc5e29ab0 [docs] Add pipelines to table (#9282) 
update pipelines
 2024-12-23 13:02:15 +05:30
Álvaro Somoza
3f17b8d56e [IP Adapter] Fix cache_dir and local_files_only for image encoder (#9272) 
initial fix
 2024-12-23 13:02:15 +05:30
Linoy Tsaban
18a3c35aac [Dreambooth flux] bug fix for dreambooth script (align with dreambooth lora) (#9257) 
* fix shape

* fix prompt encoding

* style

* fix device

* add comment
 2024-12-23 13:02:15 +05:30
YiYi Xu
9c185d8d84 refactor 3d rope for cogvideox (#9269) 
* refactor 3d rope

* repeat -> expand
 2024-12-23 13:02:15 +05:30
王奇勋
cfd12e2ece [Flux] Support Union ControlNet (#9175) 
* refactor
---------

Co-authored-by: haofanwang <haofanwang.ai@gmail.com>
 2024-12-23 13:02:15 +05:30
yangpei-comp
5a9de1993c Bugfix in pipeline_kandinsky2_2_combined.py: Image type check mismatch (#9256) 
Update pipeline_kandinsky2_2_combined.py

Bugfix on image type check mismatch
 2024-12-23 13:02:15 +05:30
Dhruv Nair
a2462e3e3a [CI] Run Fast + Fast GPU Tests on release branches. (#9255) 
* update

* update
 2024-12-23 13:02:15 +05:30
Dhruv Nair
47ab1f5089 [Single File] Add Flux Pipeline Support (#9244) 
update
 2024-12-23 13:02:15 +05:30
Dhruv Nair
9483ab4040 [Single File] Support loading Comfy UI Flux checkpoints (#9243) 
update
 2024-12-23 13:02:15 +05:30
Sayak Paul
f24f48340f [Core] fuse_qkv_projection() to Flux (#9185) 
* start fusing flux.

* test

* finish fusion

* fix-copues
 2024-12-23 13:02:15 +05:30
zR
977afbe83b Cogvideox-5B Model adapter change (#9203) 
* draft of embedding

---------

Co-authored-by: Aryan <aryan@huggingface.co>
 2024-12-23 13:02:15 +05:30
YiYi Xu
51317a8ecd fix _identify_model_variants (#9247) 
merge now, will add/fix test next
 2024-12-23 13:02:15 +05:30
Elias Rad
de303b9a15 Docs fix spelling issues (#9219) 
* fix PHILOSOPHY.md

* fix CONTRIBUTING.md

* fix tutorial_overview.md

* fix stable_diffusion.md

* Update tutorial_overview.md
 2024-12-23 13:02:15 +05:30
Aryan
7a1d37d430 [tests] fix broken xformers tests (#9206) 
* fix xformers tests

* remove unnecessary modifications to cogvideox tests

* update
 2024-12-23 13:02:15 +05:30
Sayak Paul
f3eacab6b9 [Flux LoRA] support parsing alpha from a flux lora state dict. (#9236) 
* support parsing alpha from a flux lora state dict.

* conditional import.

* fix breaking changes.

* safeguard alpha.

* fix
 2024-12-23 13:02:15 +05:30
Sayak Paul
7fb2860002 [Core] Tear apart from_pretrained() of DiffusionPipeline (#8967) 
* break from_pretrained part i.

* part ii.

* init_kwargs

* remove _fetch_init_kwargs

* type annotation

* dtyle

* switch to _check_and_update_init_kwargs_for_missing_modules.

* remove _check_and_update_init_kwargs_for_missing_modules.

* use pipeline_loading_kwargs.

* remove _determine_current_device_map.

* remove _filter_null_components.

* device_map fix.

* fix _update_init_kwargs_with_connected_pipeline.

* better handle custom pipeline.

* explain _maybe_raise_warning_for_inpainting.

* add example for model variant.

* fix
 2024-12-23 13:02:15 +05:30
Steven Liu
8006b4f307 [docs] Network alpha docstring (#9238) 
fix docstring

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
 2024-12-23 13:02:15 +05:30
YiYi Xu
6bea1304a8 Flux followup (#9074) 
* refactor rotary embeds

* adding jsmidt as co-author of this PR for https://github.com/huggingface/diffusers/pull/9133

---------

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
Co-authored-by: Joseph Smidt <josephsmidt@gmail.com>
 2024-12-23 13:02:15 +05:30
satani99
8a17331c29 Add StableDiffusionXLControlNetPAGImg2ImgPipeline (#8990) 
* Added pad controlnet sdxl img2img pipeline

---------

Co-authored-by: YiYi Xu <yixu310@gmail.com>
 2024-12-23 13:02:15 +05:30
Dhruv Nair
a8cea1fe4e [Single File] Fix configuring scheduler via legacy kwargs (#9229) 
update
 2024-12-23 13:02:15 +05:30
YiYi Xu
00f67807a5 fix a regression in is_safetensors_compatible (#9234) 
fix
 2024-12-23 13:02:15 +05:30
Vinh H. Pham
e14f476dd1 StableDiffusionLatentUpscalePipeline - positive/negative prompt embeds support (#8947) 
* make latent upscaler accept prompt embeds

---------

Co-authored-by: Dhruv Nair <dhruv.nair@gmail.com>
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
Co-authored-by: YiYi Xu <yixu310@gmail.com>
 2024-12-23 13:02:15 +05:30
Sangwon Lee
d9e3c812e0 Fix StableDiffusionXLPAGInpaintPipeline (#9128) 2024-12-23 13:02:15 +05:30
Disty0
74dfaee0b9 Custom sampler support for Stable Cascade Decoder (#9132) 
Custom sampler support Stable Cascade Decoder
 2024-12-23 13:02:15 +05:30
Vishnu V Jaddipal
c9a9cfa6e4 Fix ``from_single_file`` for xl_inpaint (#9054) 2024-12-23 13:02:15 +05:30
Dhruv Nair
240c1135f6 Remove M1 runner from Nightly Test (#9193) 
* update

* update
 2024-12-23 13:02:15 +05:30
Leo Jiang
ab9966f6ae Fix dtype error for StableDiffusionXL (#9217) 
Fix dtype error

Co-authored-by: 蒋硕 <jiangshuo9@h-partners.com>
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
 2024-12-23 13:02:15 +05:30