LLM/diffusers
1
0
Fork 0
mirror of https://github.com/huggingface/diffusers.git synced 2026-02-13 22:35:37 +08:00
Code Issues Packages Projects Releases Wiki Activity
4,913 Commits 744 Branches 94 Tags
4c5082ca685d1dd0a24907890da170e258e5cd01
Commit Graph

4913 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Yuxuan.Zhang
4ea9e828b1 CogView3Plus DiT (#9570) 
* merge 9588

* max_shard_size="5GB" for colab running

* conversion script updates; modeling test; refactor transformer

* make fix-copies

* Update convert_cogview3_to_diffusers.py

* initial pipeline draft

* make style

* fight bugs 🐛🪳

* add example

* add tests; refactor

* make style

* make fix-copies

* add co-author

YiYi Xu <yixu310@gmail.com>

* remove files

* add docs

* add co-author

Co-Authored-By: YiYi Xu <yixu310@gmail.com>

* fight docs

* address reviews

* make style

* make model work

* remove qkv fusion

* remove qkv fusion tets

* address review comments

* fix make fix-copies error

* remove None and TODO

* for FP16(draft)

* make style

* remove dynamic cfg

* remove pooled_projection_dim as a parameter

* fix tests

---------

Co-authored-by: Aryan <aryan@huggingface.co>
Co-authored-by: YiYi Xu <yixu310@gmail.com>
 2024-12-23 13:02:17 +05:30
Ryan Lin
0e332644bc Flux - soft inpainting via differential diffusion (#9268) 
* Flux - soft inpainting via differential diffusion

* .

* track changes to FluxInpaintPipeline

* make mask arrangement simplier

* make style

---------

Co-authored-by: YiYi Xu <yixu310@gmail.com>
Co-authored-by: Álvaro Somoza <asomoza@users.noreply.github.com>
Co-authored-by: asomoza <somoza.alvaro@gmail.com>
 2024-12-23 13:02:17 +05:30
Sayak Paul
6b5d5218cb [Tests] increase transformers version in test_low_cpu_mem_usage_with_loading (#9662) 
increase transformers version in test_low_cpu_mem_usage_with_loading
 2024-12-23 13:02:17 +05:30
Jinzhe Pan
cb671beb5c [docs] Fix xDiT doc image damage (#9655) 
* docs: fix xDiT doc image damage

* doc: move xdit images to hf dataset

---------

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
 2024-12-23 13:02:17 +05:30
Sayak Paul
15bdb8a09f [CI] replace ubuntu version to 22.04. (#9656) 
replace ubuntu version to 22.04.
 2024-12-23 13:02:17 +05:30
hlky
ca7a9bbf03 FluxMultiControlNetModel (#9647) 2024-12-23 13:02:17 +05:30
M Saqlain
1c038d8104 Add Differential Diffusion to Kolors (#9423) 
* Added diff diff support for kolors img2img

* Fized relative imports

* Fized relative imports

* Added diff diff support for Kolors

* Fized import issues

* Added map

* Fized import issues

* Fixed naming issues

* Added diffdiff support for Kolors img2img pipeline

* Removed example docstrings

* Added map input

* Updated latents

Co-authored-by: Álvaro Somoza <asomoza@users.noreply.github.com>

* Updated `original_with_noise`

Co-authored-by: Álvaro Somoza <asomoza@users.noreply.github.com>

* Improved code quality

---------

Co-authored-by: Álvaro Somoza <asomoza@users.noreply.github.com>
 2024-12-23 13:02:16 +05:30
GSSun
67c0df279d fix IsADirectoryError when running the training code for sd3_dreambooth_lora_16gb.ipynb (#9634) 
Add files via upload

fix IsADirectoryError when running the training code
 2024-12-23 13:02:16 +05:30
Subho Ghosh
827dd12311 flux controlnet control_guidance_start and control_guidance_end implement (#9571) 
* flux controlnet control_guidance_start and control_guidance_end implement

* minor fix - added docstrings, consistent controlnet scale flux and SD3
 2024-12-23 13:02:16 +05:30
Sayak Paul
3a508bd89a [LoRA] fix dora test to catch the warning properly. (#9627) 
fix dora test.
 2024-12-23 13:02:16 +05:30
Pakkapon Phongthawee
168a50a892 make controlnet support interrupt (#9620) 
* make controlnet support interrupt

* remove white space in controlnet interrupt
 2024-12-23 13:02:16 +05:30
SahilCarterr
1d278abe60 add PAG support for SD Img2Img (#9463) 
* added pag to sd img2img pipeline


---------

Co-authored-by: YiYi Xu <yixu310@gmail.com>
 2024-12-23 13:02:16 +05:30
Sayak Paul
c8d7ed6966 [LoRA] allow loras to be loaded with low_cpu_mem_usage. (#9510) 
* allow loras to be loaded with low_cpu_mem_usage.

* add flux support but note https://github.com/huggingface/diffusers/pull/9510\#issuecomment-2378316687

* low_cpu_mem_usage.

* fix-copies

* fix-copies again

* tests

* _LOW_CPU_MEM_USAGE_DEFAULT_LORA

* _peft_version default.

* version checks.

* version check.

* version check.

* version check.

* require peft 0.13.1.

* explicitly specify low_cpu_mem_usage=False.

* docs.

* transformers version 4.45.2.

* update

* fix

* empty

* better name initialize_dummy_state_dict.

* doc todos.

* Apply suggestions from code review

Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>

* style

* fix-copies

---------

Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
 2024-12-23 13:02:16 +05:30
Yijun Lee
6a5366cd56 refac/pipeline_output (#9582) 2024-12-23 13:02:16 +05:30
sanaka
752f5c215d Fix the bug that joint_attention_kwargs is not passed to the FLUX's transformer attention processors (#9517) 
* Update transformer_flux.py
 2024-12-23 13:02:16 +05:30
v2ray
f17a9d2075 Fixed noise_pred_text referenced before assignment. (#9537) 
* Fixed local variable noise_pred_text referenced before assignment when using PAG with guidance scale and guidance rescale at the same time.

* Fixed style.

* Made returning text pred noise an argument.
 2024-12-23 13:02:16 +05:30
Sayak Paul
504535e8ef [LoRA] Handle DoRA better (#9547) 
* handle dora.

* print test

* debug

* fix

* fix-copies

* update logits

* add warning in the test.

* make is_dora check consistent.

* fix-copies
 2024-12-23 13:02:16 +05:30
glide-the
d230b98128 fix: CogVideox train dataset _preprocess_data crop video (#9574) 
* Removed int8 to float32 conversion (`* 2.0 - 1.0`) from `train_transforms` as it caused image overexposure.

Added `_resize_for_rectangle_crop` function to enable video cropping functionality. The cropping mode can be configured via `video_reshape_mode`, supporting options: ['center', 'random', 'none'].

* The number 127.5 may experience precision loss during division operations.

* wandb request pil image Type

* Resizing bug

* del jupyter

* make style

* Update examples/cogvideo/README.md

* make style

---------

Co-authored-by: --unset <--unset>
Co-authored-by: Aryan <aryan@huggingface.co>
 2024-12-23 13:02:16 +05:30
Sayak Paul
83b656d5b2 Update distributed_inference.md to include transformer.device_map (#9553) 
* Update distributed_inference.md to include `transformer.device_map`

* Update docs/source/en/training/distributed_inference.md

Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>

---------

Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
 2024-12-23 13:02:16 +05:30
Eliseu Silva
252d0ed510 Fix for use_safetensors parameters, allow use of parameter on loading submodels (#9576) (#9587) 
* Fix for use_safetensors parameters, allow use of parameter on loading submodels (#9576)
 2024-12-23 13:02:16 +05:30
Yijun Lee
79798698fb refac: docstrings in import_utils.py (#9583) 
* refac: docstrings in import_utils.py

* Update import_utils.py
 2024-12-23 13:02:16 +05:30
captainzz
c746126c37 fix vae dtype when accelerate config using --mixed_precision="fp16" (#9601) 
* fix vae dtype when accelerate config using --mixed_precision="fp16"

* Add param for upcast vae
 2024-12-23 13:02:16 +05:30
Sayak Paul
12fec91937 [Chore] add a note on the versions in Flux LoRA integration tests (#9598) 
add a note on the versions.
 2024-12-23 13:02:16 +05:30
Clem
4528d4ad91 fix xlabs FLUX lora conversion typo (#9581) 
* fix startswith syntax in xlabs lora conversion

* Trigger CI

https://github.com/huggingface/diffusers/pull/9581#issuecomment-2395530360
 2024-12-23 13:02:16 +05:30
YiYi Xu
1800f5ae89 [sd3] make sure height and size are divisible by 16 (#9573) 
* check size

* up
 2024-12-23 13:02:16 +05:30
Xiangchendong
bdeff1ec64 fix cogvideox autoencoder decode (#9569) 
Co-authored-by: Aryan <aryan@huggingface.co>
 2024-12-23 13:02:16 +05:30
Darren Hsu
d3f668a45d Support bfloat16 for Upsample2D (#9480) 
* Support bfloat16 for Upsample2D

* Add test and use is_torch_version

* Resolve comments and add decorator

* Simplify require_torch_version_greater_equal decorator

* Run make style

---------

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
Co-authored-by: YiYi Xu <yixu310@gmail.com>
 2024-12-23 13:02:16 +05:30
JuanCarlosPi
91360def68 Add PAG support to StableDiffusionControlNetPAGInpaintPipeline (#8875) 
* Add pag to controlnet inpainting pipeline


---------

Co-authored-by: YiYi Xu <yixu310@gmail.com>
 2024-12-23 13:02:16 +05:30
hlky
4548843ba7 Add beta sigmas to other schedulers and update docs (#9538) 2024-12-23 13:02:16 +05:30
Sayak Paul
35d704ca70 [LoRA] support Kohya Flux LoRAs that have text encoders as well (#9542) 
* support kohya flux loras that have tes.
 2024-12-23 13:02:16 +05:30
Sayak Paul
df90ea984a [chore] fix: retain memory utility. (#9543) 
* fix: retain memory utility.

* fix

* quality

* free_memory.
 2024-12-23 13:02:16 +05:30
Anand Kumar
91d4b38e0a [train_instruct_pix2pix.py]Fix the LR schedulers when num_train_epochs is passed in a distributed training env (#9316) 
Fixed pix2pix lr scheduler

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
 2024-12-23 13:02:16 +05:30
Aryan
bb43d9933c [refactor] remove conv_cache from CogVideoX VAE (#9524) 
* remove conv cache from the layer and pass as arg instead

* make style

* yiyi's cleaner implementation

Co-Authored-By: YiYi Xu <yixu310@gmail.com>

* sayak's compiled implementation

Co-Authored-By: Sayak Paul <spsayakpaul@gmail.com>

---------

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
Co-authored-by: YiYi Xu <yixu310@gmail.com>
 2024-12-23 13:02:16 +05:30
Sayak Paul
2a38836db8 [Core] fix variant-identification. (#9253) 
* fix variant-idenitification.

* fix variant

* fix sharded variant checkpoint loading.

* Apply suggestions from code review

* fixes.

* more fixes.

* remove print.

* fixes

* fixes

* comments

* fixes

* apply suggestions.

* hub_utils.py

* fix test

* updates

* fixes

* fixes

* Apply suggestions from code review

Co-authored-by: YiYi Xu <yixu310@gmail.com>

* updates.

* removep patch file.

---------

Co-authored-by: YiYi Xu <yixu310@gmail.com>
 2024-12-23 13:02:16 +05:30
Sayak Paul
9405b1a2e4 [Tests] [LoRA] clean up the serialization stuff. (#9512) 
* clean up the serialization stuff.

* better
 2024-12-23 13:02:16 +05:30
PromeAI
19399207ae [examples] add train flux-controlnet scripts in example. (#9324) 
* add train flux-controlnet scripts in example.

* fix error

* fix subfolder error

* fix preprocess error

* Update examples/controlnet/README_flux.md

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>

* Update examples/controlnet/README_flux.md

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>

* fix readme

* fix note error

* add some Tutorial for deepspeed

* fix some Format Error

* add dataset_path example

* remove print, add guidance_scale CLI, readable apply

* Update examples/controlnet/README_flux.md

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>

* update,push_to_hub,save_weight_dtype,static method,clear_objs_and_retain_memory,report_to=wandb

* add push to hub in readme

* apply weighting schemes

* add note

* Update examples/controlnet/README_flux.md

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>

* make code style and quality

* fix some unnoticed error

* make code style and quality

* add example controlnet in readme

* add test controlnet

* rm Remove duplicate notes

* Fix formatting errors

* add new control image

* add model cpu offload

* update help for adafactor

* make quality & style

* make quality and style

* rename flux_controlnet_model_name_or_path

* fix back src/diffusers/pipelines/flux/pipeline_flux_controlnet.py

* fix dtype error by pre calculate text emb

* rm image save

* quality fix

* fix test

* fix tiny flux train error

* change report to to tensorboard

* fix save name error when test

* Fix shrinking errors

---------

Co-authored-by: YiYi Xu <yixu310@gmail.com>
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
Co-authored-by: Your Name <you@example.com>
 2024-12-23 13:02:16 +05:30
Sayak Paul
33a77c6a99 [LoRA] make set_adapters() method more robust. (#9535) 
* make set_adapters() method more robust.

* remove patch

* better and concise code.

* Update src/diffusers/loaders/lora_base.py

Co-authored-by: YiYi Xu <yixu310@gmail.com>

---------

Co-authored-by: YiYi Xu <yixu310@gmail.com>
 2024-12-23 13:02:16 +05:30
Aryan
f018acdb11 [bug] Precedence of operations in VAE should be slicing -> tiling (#9342) 
* bugfix: precedence of operations should be slicing -> tiling

* fix typo

* fix another typo

* deprecate current implementation of tiled_encode and use new impl

* Update src/diffusers/models/autoencoders/autoencoder_kl.py

Co-authored-by: YiYi Xu <yixu310@gmail.com>

* Update src/diffusers/models/autoencoders/autoencoder_kl.py

---------

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
Co-authored-by: YiYi Xu <yixu310@gmail.com>
 2024-12-23 13:02:16 +05:30
Álvaro Somoza
5cef1c58bf [Tests] Fix ChatGLMTokenizer (#9536) 
fix
 2024-12-23 13:02:16 +05:30
YiYi Xu
143fa19bbf flux controlnet fix (control_modes batch & others) (#9507) 
* flux controlnet mode to take into account batch size

* incorporate yiyixuxu's suggestions (cleaner logic) as well as clean up control mode handling for multi case

* fix

* fix use_guidance when controlnet is a multi and does not have config

---------

Co-authored-by: Christopher Beckham <christopher.j.beckham@gmail.com>
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
 2024-12-23 13:02:16 +05:30
hlky
a4baaae02b [Schedulers] Add beta sigmas / beta noise schedule (#9509) 
Add beta sigmas / beta noise schedule
 2024-12-23 13:02:16 +05:30
v2ray
6f9f26f8f8 [Doc] Improved level of clarity for latents_to_rgb. (#9529) 
Fixed latents_to_rgb doc.

Co-authored-by: Álvaro Somoza <asomoza@users.noreply.github.com>
 2024-12-23 13:02:16 +05:30
YiYi Xu
4ed3249840 update get_parameter_dtype (#9526) 
* up

* Update src/diffusers/models/modeling_utils.py

Co-authored-by: Aryan <aryan@huggingface.co>

---------

Co-authored-by: Aryan <aryan@huggingface.co>
 2024-12-23 13:02:16 +05:30
Steven Liu
b142eea26d [docs] Model sharding (#9521) 
* flux shard

* feedback
 2024-12-23 13:02:16 +05:30
Lee Penkman
c45940d23b Update community_projects.md (#9266) 2024-12-23 13:02:16 +05:30
Sayak Paul
d8244b52a9 [Community Pipeline] Batched implementation of Flux with CFG (#9513) 
* batched implementation of flux cfg.

* style.

* readme

* remove comments.
 2024-12-23 13:02:16 +05:30
hlky
0e22145b93 Add exponential sigmas to other schedulers and update docs (#9518) 2024-12-23 13:02:16 +05:30
YiYi Xu
30dd52ce90 a few fix for SingleFile tests (#9522) 
* update sd15 repo

* update more
 2024-12-23 13:02:16 +05:30
Sayak Paul
eec17ced14 [CI] allow faster downloads from the Hub in CI. (#9478) 
* allow faster downloads from the Hub in CI.

* HF_HUB_ENABLE_HF_TRANSFER: 1

* empty

* empty

* remove ENV HF_HUB_ENABLE_HF_TRANSFER=1.

* empty
 2024-12-23 13:02:16 +05:30
LukeLin
775f163ebc [Doc] Fix path and and also import imageio (#9506) 
* Fix bug

* import imageio
 2024-12-23 13:02:16 +05:30