LLM/diffusers
1
0
Fork 0
mirror of https://github.com/huggingface/diffusers.git synced 2026-02-19 17:30:40 +08:00
Code Issues Packages Projects Releases Wiki Activity
4,913 Commits 747 Branches 94 Tags
4c5082ca685d1dd0a24907890da170e258e5cd01
Commit Graph

4913 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Sayak Paul
57a3be2e28 [Examples] fix: prior preservation setting in DreamBooth LoRA SDXL script. (#7242) 
fix: prior preservation setting in DreamBooth LoRA SDXL script.

Co-authored-by: Linoy Tsaban <57615435+linoytsaban@users.noreply.github.com>
 2024-12-23 13:02:08 +05:30
Rinne
7d8f13b5ad fix: remove duplicated code in TemporalBasicTransformerBlock. (#7212) 
fix: remove duplicate code in TemporalBasicTransformerBlock.

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
 2024-12-23 13:02:08 +05:30
Linoy Tsaban
7c36b6d3df add DoRA training feature to sdxl dreambooth lora script (#7235) 
* dora in canonical script

* add mention of DoRA to readme
 2024-12-23 13:02:08 +05:30
Paakhhi
50b6c20e06 Refactor Prompt2Prompt: Inherit from DiffusionPipeline (#7211) 
refactor: inherit from DiffusionPipeline instead of StableDiffusionPipeline
 2024-12-23 13:02:08 +05:30
Sayak Paul
cecda32a6b [Core] move out the utilities from pipeline_utils.py (#7234) 
move out the utilities from pipeline_utils.py
 2024-12-23 13:02:08 +05:30
Nate Landman
350d21bcbe Update train_dreambooth_lora_sdxl_advanced.py (#7227) 
adding the type gives you

```
TypeError: _StoreTrueAction.__init__() got an unexpected keyword argument 'type'
```
 2024-12-23 13:02:08 +05:30
Kashif Rasul
48d4cc94b0 [Pipiline] Wuerstchen v3 aka Stable Cascasde pipeline (#6487) 
* initial diffNext v3

* move to v3 folder

* imports

* dry up the unets

* no switch_level

* fix init

* add switch_level tp config

* Fixed some things

* Added pooled text embeddings

* Initial work on adding image encoder

* changes from @dome272

* Stuff for the image encoder processing and variable naming in decoder

* fix arg name

* inference fixes

* inference fixes

* default TimestepBlock without conds

* c_skip=0 by default

* fix bfloat16 to cpu

* use config

* undo temp change

* fix gen_c_embeddings args

* change text encoding

* text encoding

* undo print

* undo .gitignore change

* Allow WuerstchenV3PriorPipeline to use the base DDPM & DDIM schedulers

* use WuerstchenV3Unet in both pipelines

* fix imports

* initial failing tests

* cleanup

* use scheduler.timesterps

* some fixes to the tests, still not fully working

* fix tests

* fix prior tests

* add dropout to the model_kwargs

* more tests passing

* update expected_slice

* initial rename

* rename tests

* rename class names

* make fix-copies

* initial docs

* autodocs

* typos

* fix arg docs

* add text_encoder info

* combined pipeline has optional image arg

* fix documentation

* Update src/diffusers/pipelines/stable_cascade/modeling_stable_cascade_common.py

Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>

* Update src/diffusers/pipelines/stable_cascade/modeling_stable_cascade_common.py

Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>

* Update src/diffusers/pipelines/stable_cascade/modeling_stable_cascade_common.py

Co-authored-by: YiYi Xu <yixu310@gmail.com>

* Update src/diffusers/pipelines/stable_cascade/modeling_stable_cascade_common.py

Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>

* Update src/diffusers/pipelines/stable_cascade/pipeline_stable_cascade.py

Co-authored-by: YiYi Xu <yixu310@gmail.com>

* Update src/diffusers/pipelines/stable_cascade/modeling_stable_cascade_common.py

Co-authored-by: YiYi Xu <yixu310@gmail.com>

* use self.config

* Update src/diffusers/pipelines/stable_cascade/modeling_stable_cascade_common.py

Co-authored-by: YiYi Xu <yixu310@gmail.com>

* c_in -> in_channels

* removed kwargs from unet's forward

* Update src/diffusers/pipelines/stable_cascade/pipeline_stable_cascade.py

Co-authored-by: YiYi Xu <yixu310@gmail.com>

* Update src/diffusers/pipelines/stable_cascade/pipeline_stable_cascade.py

Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>

* remove older callback api

* removed kwargs and fixed decoder guidance > 1

* decoder takes emeds

* check and use image_embeds

* fixed all but one decoder test

* fix decoder tests

* update callback api

* fix some more combined tests

* push combined pipeline

* initial docs

* fix doc_string

* update combined api

* no test_callback_inputs test for combined pipeline

* add optional components

* fix ordering of components

* fix combined tests

* update convert script

* Update src/diffusers/pipelines/stable_cascade/pipeline_stable_cascade_prior.py

Co-authored-by: YiYi Xu <yixu310@gmail.com>

* Update src/diffusers/pipelines/stable_cascade/pipeline_stable_cascade_prior.py

Co-authored-by: YiYi Xu <yixu310@gmail.com>

* Update src/diffusers/pipelines/stable_cascade/pipeline_stable_cascade_prior.py

Co-authored-by: YiYi Xu <yixu310@gmail.com>

* fix imports

* move effnet out of deniosing loop

* prompt_embeds_pooled only when doing guidance

* Fix repeat shape

* move StableCascadeUnet to models/unets/

* more descriptive names

* converted when numpy()

* StableCascadePriorPipelineOutput docs

* rename StableCascadeUNet

* add slow tests

* fix slow tests

* update

* update

* updated model_path

* add args for weights

* set push_to_hub to false

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

---------

Co-authored-by: Dominic Rampas <d6582533@gmail.com>
Co-authored-by: Pablo Pernias <pablo@pernias.com>
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
Co-authored-by: YiYi Xu <yixu310@gmail.com>
Co-authored-by: 99991 <99991@users.noreply.github.com>
Co-authored-by: Dhruv Nair <dhruv.nair@gmail.com>
 2024-12-23 13:02:08 +05:30
Jinay Jain
8b0d08bb4c [bug] Fix float/int guidance scale not working in StableVideoDiffusionPipeline (#7143) 
* [bug] Fix float/int guidance scale not working in `StableVideoDiffusionPipeline`

* Add test to disable CFG on SVD

---------

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
 2024-12-23 13:02:08 +05:30
bram-w
713bd43037 SDXL Turbo support and example launch (#6473) 
* support and example launch for sdxl turbo

* White space fixes

* Trailing whitespace character

* ruff format

* fix guidance_scale and steps for turbo mode

---------

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
Co-authored-by: Radames Ajna <radamajna@gmail.com>
 2024-12-23 13:02:08 +05:30
Michael
6568ee3076 add TCD Scheduler (#7174) 
* add: support TCD scheduler


---------

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
 2024-12-23 13:02:08 +05:30
iczaw
1c7084b305 [Community] PromptDiffusion Pipeline (#6752) 
* Create promptdiffusioncontrolnet.py

* Update __init__.py

Added PromptDiffusionControlNetModel

* Update __init__.py

Added PromptDiffusionControlNetModel

* Update promptdiffusioncontrolnet.py

* Create pipeline_prompt_diffusion.py

Added Prompt Diffusion pipeline.

* Create convert_original_promptdiffusion_to_diffusers.py

* Update convert_from_ckpt.py

Added download_promptdiffusion_from_original_ckpt, convert_promptdiffusion_checkpoint

* Update promptdiffusioncontrolnet.py

* Update pipeline_prompt_diffusion.py

* Update README.md

* Update pipeline_prompt_diffusion.py

* Delete src/diffusers/models/promptdiffusioncontrolnet.py

* Update __init__.py

* Update __init__.py

* Delete scripts/convert_original_promptdiffusion_to_diffusers.py

* Update convert_from_ckpt.py

* Update README.md

* Delete examples/community/pipeline_prompt_diffusion.py

* Create README.md

* Create promptdiffusioncontrolnet.py

* Create convert_original_promptdiffusion_to_diffusers.py

* Create pipeline_prompt_diffusion.py

* Update README.md

* Update pipeline_prompt_diffusion.py

* Update README.md

* Update pipeline_prompt_diffusion.py

* Update convert_original_promptdiffusion_to_diffusers.py

* Update promptdiffusioncontrolnet.py

* Update README.md

---------

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
 2024-12-23 13:02:08 +05:30
Sayak Paul
55aea75371 [Core] errors should be caught as soon as possible. (#7203) 
errors should be caught as soon as possible.
 2024-12-23 13:02:08 +05:30
Vinh H. Pham
eb3c60cbde [Docs] Update callback.md code example (#7150) 
Update callback.md

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
 2024-12-23 13:02:08 +05:30
Aryan
52e8fe6816 [docs] Improve SVD pipeline docs (#7087) 
* update svd docs

* fix example doc string

* update return type hints/docs

* update type hints

* Fix typos in pipeline_stable_video_diffusion.py

* make style && make fix-copies

* Update src/diffusers/pipelines/stable_video_diffusion/pipeline_stable_video_diffusion.py

Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>

* Update src/diffusers/pipelines/stable_video_diffusion/pipeline_stable_video_diffusion.py

Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>

* update based on suggestion

---------

Co-authored-by: M. Tolga Cangöz <mtcangoz@gmail.com>
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
 2024-12-23 13:02:08 +05:30
Thiago Crepaldi
20c8000ff7 Enable PyTorch's FakeTensorMode for EulerDiscreteScheduler scheduler (#7151) 
* Enable FakeTensorMode for EulerDiscreteScheduler scheduler

PyTorch's FakeTensorMode does not support `.numpy()` or `numpy.array()`
calls.

This PR replaces `sigmas` numpy tensor by a PyTorch tensor equivalent

Repro

```python
with torch._subclasses.FakeTensorMode() as fake_mode, ONNXTorchPatcher():
    fake_model = DiffusionPipeline.from_pretrained(model_name, low_cpu_mem_usage=False)
```

that otherwise would fail with
`RuntimeError: .numpy() is not supported for tensor subclasses.`

* Address comments
 2024-12-23 13:02:08 +05:30
M. Tolga Cangöz
06de6e0b0f Fix typos (#7181) 
* Fix typos

* Fix typos

* Fix typos and update documentation in lora.md
 2024-12-23 13:02:08 +05:30
fpgaminer
cd5863dcdf Fix: UNet2DModel::__init__ type hints; fixes issue #4806 (#7175) 
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
 2024-12-23 13:02:08 +05:30
YiYi Xu
c88ad1309d fix a bug in from_config (#7192) 
* fix

* fix

* update comment

---------

Co-authored-by: yiyixuxu <yixu310@gmail,com>
 2024-12-23 13:02:08 +05:30
Linoy Tsaban
e5f4534805 [training scripts] add tags of diffusers-training (#7206) 
* add tags for diffusers training

* add tags for diffusers training

* add tags for diffusers training

* add tags for diffusers training

* add tags for diffusers training

* add tags for diffusers training

* add dora tags for drambooth lora scripts

* style
 2024-12-23 13:02:08 +05:30
Dhruv Nair
7eecc64f9b FIx torch and cuda version in ONNX tests (#7164) 
update

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
 2024-12-23 13:02:08 +05:30
Linoy Tsaban
5b2ab4c269 [advanced dreambooth lora sdxl] add DoRA training feature (#7072) 
* add is_dora arg

* style

* add dora training feature to sd 1.5 script

* added notes about DoRA training

---------

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
 2024-12-23 13:02:08 +05:30
Sayak Paul
48a9e62978 Update requirements.txt to remove huggingface-cli (#7202) 
Internal message: https://huggingface.slack.com/archives/C03Q18WK18T/p1709529892062479
 2024-12-23 13:02:08 +05:30
Vinh H. Pham
d6bf73aaf5 adding callback_on_step_end for StableDiffusionLDM3DPipeline (#7149) 
* refactor callback

* run make style

* add copy

---------

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
 2024-12-23 13:02:08 +05:30
Sayak Paul
ba756dd5ea [Docs] more elaborate example for peft torch.compile (#7161) 
more elaborate example for peft torch.compile
 2024-12-23 13:02:08 +05:30
Sayak Paul
0e6ce9dad9 feat: add ip adapter benchmark (#6936) 
* feat: add ip adapter benchmark

* sdxl support too.

* Empty-Commit
 2024-12-23 13:02:08 +05:30
Álvaro Somoza
ec8dabf5c9 [ip-adapter] fix problem using embeds with the plus version of ip adapters (#7189) 
* initial

* check_inputs fix to the rest of pipelines

* add fix for no cfg too

* use of variable

---------

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
 2024-12-23 13:02:08 +05:30
Junsong Chen
ac66c913c7 Fix PixArt 256px inference (#6789) 
* feat 256px diffusers inference bug

* change the max_length of T5 to pipeline config file

* fix bug in convert_pixart_alpha_to_diffusers.py

* Update scripts/convert_pixart_alpha_to_diffusers.py

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>

* remove multi_scale_train parser

* Update src/diffusers/pipelines/pixart_alpha/pipeline_pixart_alpha.py

Co-authored-by: YiYi Xu <yixu310@gmail.com>

* Update src/diffusers/pipelines/pixart_alpha/pipeline_pixart_alpha.py

Co-authored-by: YiYi Xu <yixu310@gmail.com>

* styling

* change `model_token_max_length` to call argument.

* Refactoring

* add: max_sequence_length to the docstring.

---------

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
Co-authored-by: YiYi Xu <yixu310@gmail.com>
 2024-12-23 13:02:08 +05:30
Sayak Paul
0c693ca240 Support EDM-style training in DreamBooth LoRA SDXL script (#7126) 
* add: dreambooth lora script for Playground v2.5

* fix: kwarg

* address suraj's comments.

* Apply suggestions from code review

Co-authored-by: Suraj Patil <surajp815@gmail.com>

* apply suraj's suggestion

* incorporate changes in the canonical script./

* tracker naming

* fix: schedule determination

* add: two simple tests

* remove playground script

* note about edm-style training

* address pedro's comments.

* address part of Suraj's comments.

* Apply suggestions from code review

Co-authored-by: Suraj Patil <surajp815@gmail.com>

* remove guidance_scale.

* use mse_loss.

* add comments for preconditioning.

* quality

* Update examples/dreambooth/train_dreambooth_lora_sdxl.py

Co-authored-by: Suraj Patil <surajp815@gmail.com>

* tackle v-pred.

* Empty-Commit

* support edm for sdxl too.

* address suraj's comments.

* Empty-Commit

---------

Co-authored-by: Suraj Patil <surajp815@gmail.com>
 2024-12-23 13:02:08 +05:30
YiYi Xu
1943ad4079 [stalebot] fix a bug (#7156) 
fix

Co-authored-by: yiyixuxu <yixu310@gmail,com>
 2024-12-23 13:02:08 +05:30
Oleh
58294f94a9 Map speedup (#6745) 
* Speed up dataset mapping

* Fix missing columns

* Remove cache files cleanup

* Update examples/text_to_image/train_text_to_image_sdxl.py

* make style

* Fix code style

* style

* Empty-Commit

---------

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
Co-authored-by: Quentin Lhoest <42851186+lhoestq@users.noreply.github.com>
Co-authored-by: Quentin Lhoest <lhoest.q@gmail.com>
 2024-12-23 13:02:08 +05:30
Sayak Paul
64694545de fix: loading problem for sdxl lora dreambooth (#7166) 2024-12-23 13:02:08 +05:30
Quentin Lhoest
5575578b38 Fix vae_encodings_fn hash in train_text_to_image_sdxl.py (#7171) 
Update train_text_to_image_sdxl.py
 2024-12-23 13:02:08 +05:30
Dhruv Nair
f1916222ef [CI] Remove max parallel flag on slow test runners (#7162) 
update
 2024-12-23 13:02:08 +05:30
Sayak Paul
1022f5a0bd Fix LCM benchmark test (#7158) 
* make workflow dispatchable.

* fix: lcm lora compile
 2024-12-23 13:02:08 +05:30
Sayak Paul
0222544269 [Urgent][Docker CI] pin uv version for now and a minor change in the Slack notification (#7155) 
pin uv version for now.
 2024-12-23 13:02:08 +05:30
Sayak Paul
63b79643ba [CI] fix path filtering in the documentation workflows (#7153) 
fix: path
 2024-12-23 13:02:08 +05:30
YiYi Xu
f38417fe30 [ip-adapter] refactor prepare_ip_adapter_image_embeds and skip load image_encoder (#7016) 
* add

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>

---------

Co-authored-by: yiyixuxu <yixu310@gmail,com>
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
 2024-12-23 13:02:08 +05:30
M. Tolga Cangöz
a3c115949d [Docs] Fix typos (#7131) 
* Add copyright notice to relevant files and fix typos

* Set `timestep_spacing` parameter of `StableDiffusionXLPipeline`'s scheduler to `'trailing'`.

* Update `StableDiffusionXLPipeline.from_single_file` by including EulerAncestralDiscreteScheduler with `timestep_spacing="trailing"` param.

* Update model loading method in SDXL Turbo documentation
 2024-12-23 13:02:08 +05:30
Dhruv Nair
91f6646f6a Fix setting fp16 dtype in AnimateDiff convert script. (#7127) 
* update

* update
 2024-12-23 13:02:08 +05:30
Sayak Paul
1e7fcd4e17 add: support for notifying the maintainers about the docker ci status. (#7113) 2024-12-23 13:02:08 +05:30
Sayak Paul
1de9fcb023 limit documentation workflow runs for relevant changes. (#7125) 2024-12-23 13:02:08 +05:30
Sayak Paul
7d1d1afab6 use uv for installing stuff in the workflows. (#7116) 
* use uv for installing stuff in the workflows.

* fix: from source installation command when using uv.

* fix uv venv issue

* edit editable installation.

* fix quality installation

* checking

* make editable.

* more check

* check

* add: export step

* venv handling.

* checking.

* fix: dependency workflows.

* peft tests.

* proper way to initialize env.

* Empty-Commit

* Empty-Commit
 2024-12-23 13:02:08 +05:30
Aryan
f95a5ceb91 [docs] unet type hints (#7134) 
update
 2024-12-23 13:02:08 +05:30
elucida
14bb45e4e8 refactor: move model helper function in pipeline to a mixin class (#6571) 
* move model helper function in pipeline to EfficiencyMixin

---------

Co-authored-by: YiYi Xu <yixu310@gmail.com>
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
 2024-12-23 13:02:08 +05:30
YiYi Xu
ad6a21585a [stalebot] don't close the issue if the stale label is removed (#7106) 
* fix

* fix

* remove stalebot's ability to close issues

---------

Co-authored-by: yiyixuxu <yixu310@gmail,com>
 2024-12-23 13:02:08 +05:30
Dhruv Nair
cda7b5a298 [Diffusers CI] Switch slow test runners (#7123) 
update
 2024-12-23 13:02:08 +05:30
Sayak Paul
8139fa70c4 fix kwarg in the SDXL LoRA DreamBooth (#7124) 
* fix kwarg

* Empty-Commit
 2024-12-23 13:02:08 +05:30
Sayak Paul
1558485d5f [Tests] make test steps dependent on certain things and general cleanup of the workflows (#7026) 
make tests conditional and other things.
 2024-12-23 13:02:08 +05:30
Beinsezii
1a368d8dc1 DPMSolverMultistep add rescale_betas_zero_snr (#7097) 
* DPMMultistep rescale_betas_zero_snr

* DPM upcast samples in step()

* DPM rescale_betas_zero_snr UT

* DPMSolverMulti move sample upcast after model convert

Avoids having to re-use the dtype.

* Add a newline for Ruff
 2024-12-23 13:02:08 +05:30
M. Tolga Cangöz
59ed616c16 [Docs] Fix typos (#7118) 
Fix typos, formatting and remove trailing whitespace
 2024-12-23 13:02:08 +05:30