LLM/diffusers
1
0
Fork 0
mirror of https://github.com/huggingface/diffusers.git synced 2026-02-14 14:55:26 +08:00
Code Issues Packages Projects Releases Wiki Activity
4,913 Commits 743 Branches 94 Tags
4c5082ca685d1dd0a24907890da170e258e5cd01
Commit Graph

4913 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Zoltan
c8f1ac211e Add vae slicing and tiling to flux pipeline (#9122) 
add vae slicing and tiling to flux pipeline

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
 2024-12-23 13:02:15 +05:30
YiYi Xu
a9460a84cf fix autopipeline for kolors img2img (#9212) 
fix
 2024-12-23 13:02:15 +05:30
Jiwook Han
523fda4722 Reflect few contributions on contribution.md that were not reflected on #8294 (#8938) 
* incorrect_number_fix

* add_TOC

* Update docs/source/ko/conceptual/contribution.md

Co-authored-by: Jihun Lim <31366038+heuristicwave@users.noreply.github.com>

* Update docs/source/ko/conceptual/contribution.md

Co-authored-by: Jihun Lim <31366038+heuristicwave@users.noreply.github.com>

* fix: manual edits

* fix: manual edtis

* fix: manual edits

* Update docs/source/ko/conceptual/contribution.md

Co-authored-by: Jihun Lim <31366038+heuristicwave@users.noreply.github.com>

* Update docs/source/ko/conceptual/contribution.md

Co-authored-by: Jihun Lim <31366038+heuristicwave@users.noreply.github.com>

* Update docs/source/ko/conceptual/contribution.md

Co-authored-by: Jihun Lim <31366038+heuristicwave@users.noreply.github.com>

* Update docs/source/ko/conceptual/contribution.md

Co-authored-by: Jihun Lim <31366038+heuristicwave@users.noreply.github.com>

* Update docs/source/ko/conceptual/contribution.md

Co-authored-by: Jihun Lim <31366038+heuristicwave@users.noreply.github.com>

* Update docs/source/ko/conceptual/contribution.md

Co-authored-by: Jihun Lim <31366038+heuristicwave@users.noreply.github.com>

* Update docs/source/ko/conceptual/contribution.md

Co-authored-by: Jihun Lim <31366038+heuristicwave@users.noreply.github.com>

* Update docs/source/ko/conceptual/contribution.md

Co-authored-by: Jihun Lim <31366038+heuristicwave@users.noreply.github.com>

* Update docs/source/ko/conceptual/contribution.md

Co-authored-by: Jihun Lim <31366038+heuristicwave@users.noreply.github.com>

* fix: manual edits

---------

Co-authored-by: Jihun Lim <31366038+heuristicwave@users.noreply.github.com>
 2024-12-23 13:02:15 +05:30
Dhruv Nair
7256e317cf [CI] Add fail-fast=False to CUDA nightly and slow tests (#9214) 
* update

* update
 2024-12-23 13:02:15 +05:30
Dhruv Nair
7078af5785 [CI] Multiple Slow Test fixes. (#9198) 
* update

* update

* update

* update
 2024-12-23 13:02:15 +05:30
Dhruv Nair
6a0eae8406 Update is_safetensors_compatible check (#8991) 
* update

* update

* update

* update

* update
 2024-12-23 13:02:15 +05:30
Wenlong Wu
d12e48c68d Add loading text inversion (#9130) 2024-12-23 13:02:15 +05:30
M Saqlain
158042c999 [Tests] Improve transformers model test suite coverage - Lumina (#8987) 
* Added test suite for lumina

* Fixed failing tests

* Improved code quality

* Added function docstrings

* Improved formatting
 2024-12-23 13:02:15 +05:30
townwish4git
8615d0fbff fix(sd3): fix deletion of text_encoders etc (#8951) 
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
Co-authored-by: YiYi Xu <yixu310@gmail.com>
 2024-12-23 13:02:15 +05:30
Tolga Cangöz
259b12a077 [Docs] Fix CPU offloading usage (#9207) 
* chore: Fix cpu offloading usage

* Trim trailing white space

* docs: update Kolors model link in kolors.md
 2024-12-23 13:02:15 +05:30
Sayak Paul
18b7ad3c42 feat: allow sharding for auraflow. (#8853) 2024-12-23 13:02:15 +05:30
Beinsezii
32ac59e455 Add Lumina T2I Auto Pipe Mapping (#8962) 2024-12-23 13:02:15 +05:30
Jianqi Pan
b989c8c3ed fix(pipeline): k sampler sigmas device (#9189) 
If Karras is not enabled, a device inconsistency error will occur. This is due to the fact that sigmas were not moved to the specified device.
 2024-12-23 13:02:15 +05:30
Álvaro Somoza
ca73c93bd2 [IP Adapter] Fix object has no attribute with image encoder (#9194) 
* fix

* apply suggestion
 2024-12-23 13:02:15 +05:30
Sayak Paul
fc0ed5ea1f [Chore] add set_default_attn_processor to pixart. (#9196) 
add set_default_attn_processor to pixart.
 2024-12-23 13:02:15 +05:30
C
2fba12df68 [Flux] Optimize guidance creation in flux pipeline by moving it outside the loop (#9153) 
* optimize guidance creation in flux pipeline by moving it outside the loop

* use torch.full instead of torch.tensor to create a tensor with a single value

---------

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
 2024-12-23 13:02:15 +05:30
Sayak Paul
7269dda005 feat: allow flux transformer to be sharded during inference (#9159) 
* feat: support sharding for flux.

* tests
 2024-12-23 13:02:15 +05:30
Dhruv Nair
1819d6db44 Small improvements for video loading (#9183) 
* update

* update
 2024-12-23 13:02:15 +05:30
Simo Ryu
38bbc97b5c Add Learned PE selection for Auraflow (#9182) 
* add pe

* Update src/diffusers/models/transformers/auraflow_transformer_2d.py

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>

* Update src/diffusers/models/transformers/auraflow_transformer_2d.py

* beauty

* retrigger ci.

---------

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
 2024-12-23 13:02:15 +05:30
dependabot[bot]
87aed8ee7b Bump jinja2 from 3.1.3 to 3.1.4 in /examples/research_projects/realfill (#7873) 
Bumps [jinja2](https://github.com/pallets/jinja) from 3.1.3 to 3.1.4.
- [Release notes](https://github.com/pallets/jinja/releases)
- [Changelog](https://github.com/pallets/jinja/blob/main/CHANGES.rst)
- [Commits](https://github.com/pallets/jinja/compare/3.1.3...3.1.4)

---
updated-dependencies:
- dependency-name: jinja2
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Dhruv Nair <dhruv.nair@gmail.com>
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
 2024-12-23 13:02:15 +05:30
dependabot[bot]
bc4ed69e2d Bump torch from 2.0.1 to 2.2.0 in /examples/research_projects/realfill (#8971) 
Bumps [torch](https://github.com/pytorch/pytorch) from 2.0.1 to 2.2.0.
- [Release notes](https://github.com/pytorch/pytorch/releases)
- [Changelog](https://github.com/pytorch/pytorch/blob/main/RELEASE.md)
- [Commits](https://github.com/pytorch/pytorch/compare/v2.0.1...v2.2.0)

---
updated-dependencies:
- dependency-name: torch
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
 2024-12-23 13:02:15 +05:30
Álvaro Somoza
275e523ef5 post release 0.30.0 (#9173) 
* post release

* fix quality
 2024-12-23 13:02:15 +05:30
Aryan
0f74c69416 [refactor] CogVideoX followups + tiled decoding support (#9150) 
* refactor context parallel cache; update torch compile time benchmark

* add tiling support

* make style

* remove num_frames % 8 == 0 requirement

* update default num_frames to original value

* add explanations + refactor

* update torch compile example

* update docs

* update

* clean up if-statements

* address review comments

* add test for vae tiling

* update docs

* update docs

* update docstrings

* add modeling test for cogvideox transformer

* make style
 2024-12-23 13:02:15 +05:30
王奇勋
d71e408d58 [FLUX] Support ControlNet (#9126) 
* cnt model

* cnt model

* cnt model

* fix Loader "Copied"

* format

* txt_ids for  multiple images

* add test and format

* typo

* Update pipeline_flux_controlnet.py

* remove

* make quality

* fix copy

* Update src/diffusers/pipelines/flux/pipeline_flux_controlnet.py

Co-authored-by: Dhruv Nair <dhruv.nair@gmail.com>

* Update src/diffusers/pipelines/flux/pipeline_flux_controlnet.py

Co-authored-by: Dhruv Nair <dhruv.nair@gmail.com>

* Update src/diffusers/pipelines/flux/pipeline_flux_controlnet.py

Co-authored-by: Dhruv Nair <dhruv.nair@gmail.com>

* Update src/diffusers/pipelines/flux/pipeline_flux_controlnet.py

Co-authored-by: Dhruv Nair <dhruv.nair@gmail.com>

* Update src/diffusers/models/controlnet_flux.py

Co-authored-by: Dhruv Nair <dhruv.nair@gmail.com>

* fix

* make copies

* test

* bs

---------

Co-authored-by: haofanwang <haofanwang.ai@gmail.com>
Co-authored-by: haofanwang <haofan@HaofandeMBP.lan>
Co-authored-by: Dhruv Nair <dhruv.nair@gmail.com>
 2024-12-23 13:02:15 +05:30
林金鹏
095393a5b8 Support SD3 controlnet inpainting (#9099) 
* add controlnet inpainting pipeline

* [SD3] add controlnet inpaint example

* update example and fix code style

* fix code style with ruff

* Update controlnet_sd3.md : add control inpaint pipeline

* Update docs/source/en/api/pipelines/controlnet_sd3.md

Co-authored-by: Aryan <contact.aryanvs@gmail.com>

* Update docs/source/en/api/pipelines/controlnet_sd3.md

Co-authored-by: Aryan <contact.aryanvs@gmail.com>

* Update docs/source/en/api/pipelines/controlnet_sd3.md

Co-authored-by: Aryan <contact.aryanvs@gmail.com>

* Update src/diffusers/pipelines/controlnet_sd3/pipeline_stable_diffusion_3_controlnet_inpainting.py

Co-authored-by: Aryan <contact.aryanvs@gmail.com>

* Update __init__.py : add sd3 control pipelines

* Update pipeline : add new param doc & check input reference.

* fix typo

* make style & make quality

* add unittest for sd3 controlnet inpaint

---------

Co-authored-by: 鹏徙 <linjinpeng.ljp@alibaba-inc.com>
Co-authored-by: Aryan <contact.aryanvs@gmail.com>
 2024-12-23 13:02:15 +05:30
Sayak Paul
0c78d2af0b Update distributed_inference.md to include a fuller example on distributed inference (#9152) 
* Update distributed_inference.md

* Update docs/source/en/training/distributed_inference.md

Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>

---------

Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
 2024-12-23 13:02:15 +05:30
Linoy Tsaban
5fd5487f18 [Flux Dreambooth LoRA] - te bug fixes & updates (#9139) 
* add requirements + fix link to bghira's guide

* text ecnoder training fixes

* text encoder training fixes

* text encoder training fixes

* text encoder training fixes

* style

* add tests

* fix encode_prompt call

* style

* unpack_latents test

* fix lora saving

* remove default val for max_sequenece_length in encode_prompt

* remove default val for max_sequenece_length in encode_prompt

* style

* testing

* style

* testing

* testing

* style

* fix sizing issue

* style

* revert scaling

* style

* style

* scaling test

* style

* scaling test

* remove model pred operation left from pre-conditioning

* remove model pred operation left from pre-conditioning

* fix trainable params

* remove te2 from casting

* transformer to accelerator

* remove prints

* empty commit
 2024-12-23 13:02:15 +05:30
Dhruv Nair
fc0f4c5eae Update Video Loading/Export to use imageio (#9094) 
* update

* update

* update

---------

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
 2024-12-23 13:02:15 +05:30
Dibbla!
d5c0d5dbba Errata - fix typo (#9100) 2024-12-23 13:02:15 +05:30
Steven Liu
052edeba21 [docs] Resolve internal links to PEFT (#9144) 
* resolve peft links

* fuse_lora
 2024-12-23 13:02:15 +05:30
Daniel Socek
e42d61e021 Fix textual inversion SDXL and add support for 2nd text encoder (#9010) 
* Fix textual inversion SDXL and add support for 2nd text encoder

Signed-off-by: Daniel Socek <daniel.socek@intel.com>

* Fix style/quality of text inv for sdxl

Signed-off-by: Daniel Socek <daniel.socek@intel.com>

---------

Signed-off-by: Daniel Socek <daniel.socek@intel.com>
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
 2024-12-23 13:02:15 +05:30
Linoy Tsaban
6e9c6a298b [Flux] Dreambooth LoRA training scripts (#9086) 
* initial commit - dreambooth for flux

* update transformer to be FluxTransformer2DModel

* update training loop and validation inference

* fix sd3->flux docs

* add guidance handling, not sure if it makes sense(?)

* inital dreambooth lora commit

* fix text_ids in compute_text_embeddings

* fix imports of static methods

* fix pipeline loading in readme, remove auto1111 docs for now

* fix pipeline loading in readme, remove auto1111 docs for now, remove some irrelevant text_encoder_3 refs

* Update examples/dreambooth/train_dreambooth_flux.py

Co-authored-by: Bagheera <59658056+bghira@users.noreply.github.com>

* fix te2 loading and remove te2 refs from text encoder training

* fix tokenizer_2 initialization

* remove text_encoder training refs from lora script (for now)

* try with vae in bfloat16, fix model hook save

* fix tokenization

* fix static imports

* fix CLIP import

* remove text_encoder training refs (for now) from lora script

* fix minor bug in encode_prompt, add guidance def in lora script, ...

* fix unpack_latents args

* fix license in readme

* add "none" to weighting_scheme options for uniform sampling

* style

* adapt model saving - remove text encoder refs

* adapt model loading - remove text encoder refs

* initial commit for readme

* Update examples/dreambooth/train_dreambooth_lora_flux.py

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>

* Update examples/dreambooth/train_dreambooth_lora_flux.py

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>

* fix vae casting

* remove precondition_outputs

* readme

* readme

* style

* readme

* readme

* update weighting scheme default & docs

* style

* add text_encoder training to lora script, change vae_scale_factor value in both

* style

* text encoder training fixes

* style

* update readme

* minor fixes

* fix te params

* fix te params

---------

Co-authored-by: Bagheera <59658056+bghira@users.noreply.github.com>
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
 2024-12-23 13:02:15 +05:30
Sayak Paul
a60eb14a5c Update README.md to include InstantID (#8770) 
Co-authored-by: YiYi Xu <yixu310@gmail.com>
 2024-12-23 13:02:15 +05:30
Monjoy Narayan Choudhury
a46c3d7f90 Add Differential Diffusion to HunyuanDiT. (#9040) 
* Add Differential Pipeline.

* Fix Styling Issue using ruff -fix

* Add details to Contributing.md

* Revert "Fix Styling Issue using ruff -fix"

This reverts commit d347de162d.

* Revert "Revert "Fix Styling Issue using ruff -fix""

This reverts commit ce7c3ff216.

* Revert README changes

* Restore README.md

* Update README.md

* Resolved Comments:

* Fix Readme based on review

* Fix formatting after make style

---------

Co-authored-by: Aryan <aryan@huggingface.co>
 2024-12-23 13:02:15 +05:30
David Steinberg
d8d8e86924 Fix a dead link (#9116) 
Co-authored-by: Aryan <aryan@huggingface.co>
 2024-12-23 13:02:15 +05:30
sayantan sadhu
23e204790d fix for lr scheduler in distributed training (#9103) 
* fix for lr scheduler in distributed training

* Fixed the recalculation of the total training step section

* Fixed lint error

---------

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
 2024-12-23 13:02:15 +05:30
Steven Liu
c690fc2635 [docs] Organize model toctree (#9118) 
* toctree

* fix
 2024-12-23 13:02:15 +05:30
zR
dbf5d348e6 Add CogVideoX text-to-video generation model (#9082) 
* add CogVideoX

---------

Co-authored-by: Aryan <aryan@huggingface.co>
Co-authored-by: sayakpaul <spsayakpaul@gmail.com>
Co-authored-by: Aryan <contact.aryanvs@gmail.com>
Co-authored-by: yiyixuxu <yixu310@gmail.com>
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
 2024-12-23 13:02:15 +05:30
Dhruv Nair
871d32eecb Freenoise change vae_batch_size to decode_chunk_size (#9110) 
* update

* update
 2024-12-23 13:02:15 +05:30
Aryan
fbb294e8e0 [feat] allow sparsectrl to be loaded from single file (#9073) 
* allow sparsectrl to be loaded with single file

* update

---------

Co-authored-by: Dhruv Nair <dhruv.nair@gmail.com>
 2024-12-23 13:02:15 +05:30
latentCall145
f771be1d7b Flux fp16 inference fix (#9097) 
* clipping for fp16

* fix typo

* added fp16 inference to docs

* fix docs typo

* include link for fp16 investigation

---------

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
 2024-12-23 13:02:15 +05:30
Aryan
b6fac9d573 [core] FreeNoise (#8948) 
* initial work draft for freenoise; needs massive cleanup

* fix freeinit bug

* add animatediff controlnet implementation

* revert attention changes

* add freenoise

* remove old helper functions

* add decode batch size param to all pipelines

* make style

* fix copied from comments

* make fix-copies

* make style

* copy animatediff controlnet implementation from #8972

* add experimental support for num_frames not perfectly fitting context length, ocntext stride

* make unet motion model lora work again based on #8995

* copy load video utils from #8972

* copied from AnimateDiff::prepare_latents

* address the case where last batch of frames does not match length of indices in prepare latents

* decode_batch_size->vae_batch_size; batch vae encode support in animatediff vid2vid

* revert sparsectrl and sdxl freenoise changes

* revert pia

* add freenoise tests

* make fix-copies

* improve docstrings

* add freenoise tests to animatediff controlnet

* update tests

* Update src/diffusers/models/unets/unet_motion_model.py

* add freenoise to animatediff pag

* address review comments

* make style

* update tests

* make fix-copies

* fix error message

* remove copied from comment

* fix imports in tests

* update

---------

Co-authored-by: Dhruv Nair <dhruv.nair@gmail.com>
 2024-12-23 13:02:15 +05:30
Sayak Paul
f35bdb6a03 fix train_dreambooth_lora_sd3.py loading hook (#9107) 2024-12-23 13:02:15 +05:30
Álvaro Somoza
3510d0ef5e [Kolors] Add PAG (#8934) 
* txt2img pag added

* autopipe added, fixed case

* style

* apply suggestions

* added fast tests, added todo tests

* revert dummy objects for kolors

* fix pag dummies

* fix test imports

* update pag tests

* add kolor pag to docs

---------

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
 2024-12-23 13:02:15 +05:30
Dhruv Nair
47874e837d [Single File] Add single file support for Flux Transformer (#9083) 
* update

* update

* update

---------

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
 2024-12-23 13:02:15 +05:30
Marc Sun
8bdafc6fc4 Fix loading sharded checkpoints when we have variants (#9061) 
* Fix loading sharded checkpoint when we have variant

* add test

* remote print

---------

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
 2024-12-23 13:02:15 +05:30
Ahn Donghoon (안동훈 / suno)
f25823781d add PAG support for Stable Diffusion 3 (#8861) 
add pag sd3


---------

Co-authored-by: HyoungwonCho <jhw9811@korea.ac.kr>
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
Co-authored-by: crepejung00 <jaewoojung00@naver.com>
Co-authored-by: YiYi Xu <yixu310@gmail.com>
Co-authored-by: Aryan <contact.aryanvs@gmail.com>
Co-authored-by: Aryan <aryan@huggingface.co>
 2024-12-23 13:02:15 +05:30
Dhruv Nair
4a91ee80c2 [Docs] Add community projects section to docs (#9013) 
* update

* update

* update
 2024-12-23 13:02:15 +05:30
Dhruv Nair
faa0826328 update 2024-12-23 13:02:15 +05:30
Vinh H. Pham
81d58eb03e [Tests] Improve transformers model test suite coverage - Hunyuan DiT (#8916) 
* add hunyuan model test

* apply suggestions

* reduce dims further

* reduce dims further

* run make style

---------

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
 2024-12-23 13:02:15 +05:30