LLM/diffusers
1
0
Fork 0
mirror of https://github.com/huggingface/diffusers.git synced 2026-02-15 07:15:42 +08:00
Code Issues Packages Projects Releases Wiki Activity
4,913 Commits 744 Branches 94 Tags
4c5082ca685d1dd0a24907890da170e258e5cd01
Commit Graph

4913 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
XCL
f488493082 [Tencent Hunyuan Team] Add Hunyuan-DiT ControlNet Inference (#8694) 
* add controlnet support

---------

Co-authored-by: xingchaoliu <xingchaoliu@tencent.com>
Co-authored-by: yiyixuxu <yixu310@gmail,com>
 2024-12-23 13:02:13 +05:30
YiYi Xu
b078f93d6a [doc] add more about from_pipe API for PAG doc (#8701) 
* add more about from_pipe API

* Update docs/source/en/using-diffusers/pag.md

* Update docs/source/en/using-diffusers/pag.md

---------

Co-authored-by: yiyixuxu <yixu310@gmail,com>
 2024-12-23 13:02:13 +05:30
Dhruv Nair
32c5ea06b5 Add decorator for compile tests (#8703) 
* update

* update

---------

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
 2024-12-23 13:02:13 +05:30
Sayak Paul
3ecdab0995 add docs on model sharding (#8658) 
* add docs on model sharding

* add entry to _toctree.

* Apply suggestions from code review

Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>

* simplify wording

* add a note on transformer library handling

* move device placement section

* Update docs/source/en/training/distributed_inference.md

Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>

---------

Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
 2024-12-23 13:02:13 +05:30
YiYi Xu
7c4736c0ff add sd3 conversion script (#8702) 
add conversion script
 2024-12-23 13:02:13 +05:30
Álvaro Somoza
89a6943efc [Docs] SD3 T5 Token limit doc (#8654) 
* doc for max_sequence_length

* better position and changed note to tip

* apply suggestions

---------

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
 2024-12-23 13:02:13 +05:30
YiYi Xu
5efc438c7e add PAG support (#7944) 
* first draft


---------

Co-authored-by: yiyixuxu <yixu310@gmail,com>
Co-authored-by: Junhwa Song <ethan9867@gmail.com>
Co-authored-by: Ahn Donghoon (안동훈 / suno) <suno.vivid@gmail.com>
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
 2024-12-23 13:02:13 +05:30
Sayak Paul
712e9b3108 [Marigold tests] add is_flaky decorator to some Marigold tests (#8696) 
okay
 2024-12-23 13:02:13 +05:30
Linoy Tsaban
f8c8daa0ae [Sd3 Dreambooth LoRA] Add text encoder training for the clip encoders (#8630) 
* add clip text-encoder training

* no dora

* text encoder traing fixes

* text encoder traing fixes

* text encoder training fixes

* text encoder training fixes

* text encoder training fixes

* text encoder training fixes

* add text_encoder layers to save_lora

* style

* fix imports

* style

* fix text encoder

* review changes

* review changes

* review changes

* minor change

* add lora tag

* style

* add readme notes

* add tests for clip encoders

* style

* typo

* fixes

* style

* Update tests/lora/test_lora_layers_sd3.py

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>

* Update examples/dreambooth/README_sd3.md

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>

* minor readme change

---------

Co-authored-by: YiYi Xu <yixu310@gmail.com>
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
 2024-12-23 13:02:13 +05:30
Sayak Paul
658f3d524b [Chore] create a utility for calculating the expected number of shards. (#8692) 
create a utility for calculating the expected number of shards.
 2024-12-23 13:02:13 +05:30
Hammond Liu
6f922c2a91 Fix redundant pipe init in sd3 lora (#8680) 
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
 2024-12-23 13:02:13 +05:30
Tolga Cangöz
58496f6ab4 Errata - Update class method convention to use cls (#8574) 
* Class methods are supposed to use `cls` conventionally

* `make style && make quality`

* An Empty commit

---------

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
 2024-12-23 13:02:13 +05:30
Steven Liu
473acb9579 [docs] Add note for float8 (#8685) 
add note
 2024-12-23 13:02:13 +05:30
Steven Liu
f5dca81b2f [docs] Fix Pillow import (#8684) 
fix import error
 2024-12-23 13:02:13 +05:30
Sayak Paul
1850ad61c4 [Docs] add note on caching in fast diffusion (#8675) 
* add note on caching in fast diffusion

* formatting

* Update docs/source/en/tutorials/fast_diffusion.md

Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>

---------

Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
 2024-12-23 13:02:13 +05:30
Tolga Cangöz
f6172748c6 Errata - Fix typos and improve style (#8571) 
* Fix typos

* Fix typos & up style

* chore: Update numbers

---------

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
 2024-12-23 13:02:13 +05:30
Tolga Cangöz
1ced1c40d8 Discourage using deprecated revision parameter (#8573) 
* Discourage using `revision`

* `make style && make quality`

* Refactor code to use 'variant' instead of 'revision'

* `revision="bf16"` -> `variant="bf16"`

---------

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
 2024-12-23 13:02:13 +05:30
Tolga Cangöz
2c56360222 Errata - Trim trailing white space in the whole repo (#8575) 
* Trim all the trailing white space in the whole repo

* Remove unnecessary empty places

* make style && make quality

* Trim trailing white space

* trim

---------

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
 2024-12-23 13:02:13 +05:30
Dong
11a890caa4 🎨 fix xl playground device (#8550) 
* 🎨 fix xl playground device

* 🎨 run `make fix-copies`

* 🎨 run `make fix-copies`

* edit xl_controlnet_img2img file

* edit playground img2img test slow

* Update tests/pipelines/stable_diffusion_xl/test_stable_diffusion_xl_img2img.py

---------

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
 2024-12-23 13:02:13 +05:30
Tolga Cangöz
c20d0ec72d Errata - Fix typos & improve contributing page (#8572) 
* Fix typos & improve contributing page

* `make style && make quality`

* fix typos

* Fix typo

---------

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
 2024-12-23 13:02:13 +05:30
Vinh H. Pham
74acd0fb64 [train_lcm_distill_lora_sdxl.py] Fix the LR schedulers when num_train_epochs is passed in a distributed training env (#8446) 
fix num_train_epochs

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
 2024-12-23 13:02:13 +05:30
drhead
33646098f5 Add extra performance features for EMAModel, torch._foreach operations and better support for non-blocking CPU offloading (#7685) 
* Add support for _foreach operations and non-blocking to EMAModel

* default foreach to false

* add non-blocking EMA offloading to SD1.5 T2I example script

* fix whitespace

* move foreach to cli argument

* linting

* Update README.md re: EMA weight training

* correct args.foreach_ema

* add tests for foreach ema

* code quality

* add foreach to from_pretrained

* default foreach false

* fix linting

---------

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
Co-authored-by: drhead <a@a.a>
 2024-12-23 13:02:13 +05:30
Haofan Wang
b4cdb1222d Update pipeline_stable_diffusion_3_controlnet.py (#8660) 
Co-authored-by: YiYi Xu <yixu310@gmail,com>
 2024-12-23 13:02:13 +05:30
Sayak Paul
c90e70dde1 [LoRA] refactor lora conversion utility. (#8295) 
* refactor lora conversion utility.

* remove error raises.

* add onetrainer support too.
 2024-12-23 13:02:13 +05:30
Álvaro Somoza
48d590de20 [SD3 LoRA] Fix list index out of range (#8584) 
* fix

* add check

* key present is checked before

* test case draft

* aply suggestions

* changed testing repo, back to old class

* forgot docstring

---------

Co-authored-by: YiYi Xu <yixu310@gmail.com>
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
 2024-12-23 13:02:13 +05:30
Sayak Paul
22ebb0ab89 [LoRA] get rid of the legacy lora remnants and make our codebase lighter (#8623) 
* get rid of the legacy lora remnants and make our codebase lighter

* fix depcrecated lora argument

* fix

* empty commit to trigger ci

* remove print

* empty
 2024-12-23 13:02:13 +05:30
YiYi Xu
af474e5d44 a few fix for shard checkpoints (#8656) 
fix

Co-authored-by: yiyixuxu <yixu310@gmail,com>
 2024-12-23 13:02:13 +05:30
Steaunk
6da31e4e56 Fix Typo in StableDiffusion3 (#8642) 
* fix typo in __call__ of pipeline_stable_diffusion_3.py

* fix typo in __call__ of pipeline_stable_diffusion_3_img2img.py

---------

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
 2024-12-23 13:02:13 +05:30
satani99
492900099c Update train_dreambooth_lora_sd3.py (#8600) 
* Update train_dreambooth_lora_sd3.py

* Update train_dreambooth_lora_sd3.py

* Update train_dreambooth_sd3.py

---------

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
 2024-12-23 13:02:13 +05:30
Sayak Paul
58913cdc94 [LoRA SD3] add support for lora fusion in sd3 (#8616) 
* add support for lora fusion in sd3

* add test to ensure fused lora and effective lora produce same outpouts
 2024-12-23 13:02:13 +05:30
Sayak Paul
2c6abb2635 [Flax tests] reduce tolerance for a flax test (#8640) 
reduce tolerance for a flax test
 2024-12-23 13:02:13 +05:30
YiYi Xu
3b539834f1 fix from_single_file for checkpoints with t5 (#8631) 
fix single file
 2024-12-23 13:02:13 +05:30
Sayak Paul
af2763cf1a Change the default weighting_scheme in the SD3 scripts (#8639) 
* change to logit_normal as the weighting scheme

* sensible default mote
 2024-12-23 13:02:13 +05:30
王奇勋
86da4dcf8e Support SD3 ControlNet and Multi-ControlNet. (#8566) 
* sd3 controlnet



---------

Co-authored-by: haofanwang <haofanwang.ai@gmail.com>
 2024-12-23 13:02:13 +05:30
Nan
b2f8a01116 [SD3] Fix mis-matched shape when num_images_per_prompt > 1 using without T5 (text_encoder_3=None) (#8558) 
* fix shape mismatch when num_images_per_prompt > 1 and text_encoder_3=None

* style

* fix copies

---------

Co-authored-by: YiYi Xu <yixu310@gmail.com>
Co-authored-by: yiyixuxu <yixu310@gmail,com>
 2024-12-23 13:02:13 +05:30
Carolinabanana
4a39745728 Fix gradient checkpointing issue for Stable Diffusion 3 (#8542) 
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
Co-authored-by: YiYi Xu <yixu310@gmail.com>
 2024-12-23 13:02:13 +05:30
Yongsen Mao
73f302aeb7 add sd1.5 compatibility to controlnet-xs and fix unused_parameters error during training (#8606) 
* add sd1.5 compatibility to controlnet-xs

* set use_linear_projection by base_block

* refine code style
 2024-12-23 13:02:13 +05:30
kkj15dk
cf272f7987 self.upsample = Upsample1D (#8580) 
Making self.upsample actually be Upsample1D
 2024-12-23 13:02:13 +05:30
Vasco Ramos
410eb1ad86 [SD3 Docs] Corrected title about loading model with T5 "without" -> "with" (#8602) 
[SD3 Docs] Corrected title about loading model with T5

Corrected the documentation title to "Loading the single file checkpoint with T5" Previously, it incorrectly stated "Loading the single file checkpoint without T5" which contradicted the code snippet showing how to load the SD3 checkpoint with the T5 model
 2024-12-23 13:02:13 +05:30
Gæros
a1c5fdd62e [LoRA] text encoder: read the ranks for all the attn modules (#8324) 
* [LoRA] text encoder: read the ranks for all the attn modules

 * In addition to out_proj, read the ranks of adapters for q_proj, k_proj, and  v_proj

 * Allow missing adapters (UNet already supports this)

* ruff format loaders.lora

* [LoRA] add tests for partial text encoders LoRAs

* [LoRA] update test_simple_inference_with_partial_text_lora to be deterministic

* [LoRA] comment justifying test_simple_inference_with_partial_text_lora

* style

---------

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
 2024-12-23 13:02:13 +05:30
Andrew Hong
63e1fcdf8d Remove underlines between badges (#8484) 
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
 2024-12-23 13:02:13 +05:30
Sayak Paul
db74292bb3 [Core] Add shift_factor to SD3 tiny autoencoder (#8618) 
* shift factor argument to tiny

* remove shift factor rejigging from the sd3 docs
 2024-12-23 13:02:13 +05:30
Álvaro Somoza
f6e4f29816 [SD3 Inference] T5 Token limit (#8506) 
* max_sequence_length for the T5

* updated img2img

* apply suggestions

---------

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
Co-authored-by: YiYi Xu <yixu310@gmail.com>
 2024-12-23 13:02:13 +05:30
Marc Sun
eaf258b461 Fix sharding when no device_map is passed (#8531) 
* Fix sharding when no device_map is passed

* style

* add tests

* align

* add docstring

* format

---------

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
 2024-12-23 13:02:13 +05:30
MaoXianXin
c8deda5834 A backslash is missing from the run command (#8471) 2024-12-23 13:02:13 +05:30
Dhruv Nair
ea1b9276e8 [Single File Loading] Handle unexpected keys in CLIP models when accelerate isn't installed. (#8462) 
* update

* update

* update

* update

* update

---------

Co-authored-by: YiYi Xu <yixu310@gmail.com>
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
 2024-12-23 13:02:13 +05:30
Álvaro Somoza
fcbb2ffac9 [SD3] TAESD3 docs (#8607) 
* tased3 docs

* apply suggestion

---------

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
 2024-12-23 13:02:13 +05:30
Sayak Paul
6ad6c79fc7 [LoRA] training fix the position of param casting when loading them (#8460) 
fix the position of param casting when loading them
 2024-12-23 13:02:13 +05:30
Sayak Paul
2fad608d8e [SD3 training] refactor the density and weighting utilities. (#8591) 
refactor the density and weighting utilities.
 2024-12-23 13:02:13 +05:30
Bagheera
acfcfd3003 SD3: update default training timestep / loss weighting distribution to logit_normal (#8592) 
Co-authored-by: bghira <bghira@users.github.com>
Co-authored-by: Kashif Rasul <kashif.rasul@gmail.com>
 2024-12-23 13:02:13 +05:30