LLM/diffusers
1
0
Fork 0
mirror of https://github.com/huggingface/diffusers.git synced 2026-02-13 22:35:37 +08:00
Code Issues Packages Projects Releases Wiki Activity
4,913 Commits 744 Branches 94 Tags
4c5082ca685d1dd0a24907890da170e258e5cd01
Commit Graph

4913 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Ethan Smith
f5d7e319c0 fix min-snr implementation (#8466) 
* fix min-snr implementation

https://github.com/kohya-ss/sd-scripts/blob/main/library/custom_train_functions.py#L66

* Update train_dreambooth.py

fix variable name mse_loss_weights

* fix divisor

* make style

---------

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
Co-authored-by: YiYi Xu <yixu310@gmail.com>
 2024-12-23 13:02:19 +05:30
hlky
bcabac2914 Add ControlNetUnion (#10131) 
* ControlNetUnion model
 2024-12-23 13:02:18 +05:30
Dhruv Nair
93dfaec4a9 [Single File] Add single file support for AutoencoderDC (#10183) 
* update

* update

* update
 2024-12-23 13:02:18 +05:30
SahilCarterr
d1411b305f Added Error when len(gligen_images ) is not equal to len(gligen_phrases) in StableDiffusionGLIGENTextImagePipeline (#10176) 
* added check value error

* fix style
 2024-12-23 13:02:18 +05:30
Jonathan Yin
d38bca7765 Fix Nonetype attribute error when loading multiple Flux loras (#10182) 
Fix Nonetype attribute error
 2024-12-23 13:02:18 +05:30
Linoy Tsaban
d6bc975868 [community pipeline rf-inversion] - fix example in doc (#10179) 
* fix example in doc

* remove redundancies

* change param
 2024-12-23 13:02:18 +05:30
Darshil Jariwala
a7ab84ea16 Add PAG Support for Stable Diffusion Inpaint Pipeline (#9386) 
* using sd inpaint pipeline and sdxl pag inpaint pipeline to add changes

* using sd inpaint pipeline and sdxl pag inpaint pipeline to add changes

* finished the call function

* added auto pipeline

* merging diffusers

* ready to test

* ready to test

* added copied from and removed unnecessary tests

* make style changes

* doc changes

* updating example doc string

* style fix

* init

* adding imports

* quality

* Update src/diffusers/pipelines/pag/pipeline_pag_sd_inpaint.py

* make

* Update tests/pipelines/pag/test_pag_sd_inpaint.py

* slice and size

* slice

---------

Co-authored-by: Darshil Jariwala <darshiljariwala@Darshils-MacBook-Air.local>
Co-authored-by: Darshil Jariwala <jariwala.darshil2002@gmail.com>
Co-authored-by: YiYi Xu <yixu310@gmail.com>
Co-authored-by: hlky <hlky@hlky.ac>
 2024-12-23 13:02:18 +05:30
Aryan
7c4bfa64b1 Flux Control LoRA (#9999) 
* update


---------

Co-authored-by: yiyixuxu <yixu310@gmail.com>
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
 2024-12-23 13:02:18 +05:30
hlky
9b031cc829 Use torch in get_3d_rotary_pos_embed/_allegro (#10161) 
Use torch in get_3d_rotary_pos_embed/_allegro
 2024-12-23 13:02:18 +05:30
Soof Golan
641eebf784 Improve post-processing performance (#10170) 
* Use multiplication instead of division
* Add fast path when denormalizing all or none of the images
 2024-12-23 13:02:18 +05:30
Linoy Tsaban
42159b7c6a [community pipeline] Add RF-inversion Flux pipeline (#9816) 
* initial commit

* update denoising loop

* fix scheduling

* style

* fix import

* fixes

* fixes

* style

* fixes

* change invert

* change denoising & check inputs

* shape & timesteps fixes

* timesteps fixes

* style

* remove redundancies

* small changes

* update documentation a bit

* update documentation a bit

* update documentation a bit

* style

* change strength param, remove redundancies

* style

* forward ode loop change

* add inversion progress bar

* fix image_seq_len

* revert to strength but == 1 by default.

* style

* add "copied from..." comments

* credit authors

* make style

* return inversion outputs without self-assigning

* adjust denoising loop to generate regular images if inverted latents are not provided

* adjust denoising loop to generate regular images if inverted latents are not provided

* fix import

* comment

* remove redundant line

* modify comment on ti

* Update examples/community/pipeline_flux_rf_inversion.py

Co-authored-by: hlky <hlky@hlky.ac>

* Update examples/community/pipeline_flux_rf_inversion.py

Co-authored-by: hlky <hlky@hlky.ac>

* Update examples/community/pipeline_flux_rf_inversion.py

Co-authored-by: hlky <hlky@hlky.ac>

* Update examples/community/pipeline_flux_rf_inversion.py

Co-authored-by: hlky <hlky@hlky.ac>

* Update examples/community/pipeline_flux_rf_inversion.py

Co-authored-by: hlky <hlky@hlky.ac>

* Update examples/community/pipeline_flux_rf_inversion.py

Co-authored-by: hlky <hlky@hlky.ac>

* Update examples/community/pipeline_flux_rf_inversion.py

Co-authored-by: hlky <hlky@hlky.ac>

* fix syntax error

---------

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
Co-authored-by: hlky <hlky@hlky.ac>
 2024-12-23 13:02:18 +05:30
Aryan
6816ee2406 [Single file] Support revision argument when loading single file config (#10168) 
update
 2024-12-23 13:02:18 +05:30
Yu Zheng
3271d389bc support sd3.5 for controlnet example (#9860) 
* support sd3.5 in controlnet

---------

Co-authored-by: YiYi Xu <yixu310@gmail.com>
 2024-12-23 13:02:18 +05:30
Juan Acevedo
24a8d2d872 Update ptxla training (#9864) 
* update ptxla example

---------

Co-authored-by: Juan Acevedo <jfacevedo@google.com>
Co-authored-by: Pei Zhang <zpcore@gmail.com>
Co-authored-by: Pei Zhang <piz@google.com>
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
Co-authored-by: Pei Zhang <pei@Peis-MacBook-Pro.local>
Co-authored-by: hlky <hlky@hlky.ac>
 2024-12-23 13:02:18 +05:30
Sayak Paul
e179be2ca3 [LoRA] depcrecate save_attn_procs(). (#10126) 
depcrecate save_attn_procs().
 2024-12-23 13:02:18 +05:30
zhangp365
1feae4160e fixed a dtype bfloat16 bug in torch_utils.py (#10125) 
* fixed a dtype bfloat16 bug in torch_utils.py

when generating 1024*1024 image with bfloat16 dtype, there is an exception:
  File "/opt/conda/lib/python3.10/site-packages/diffusers/utils/torch_utils.py", line 107, in fourier_filter
    x_freq = fftn(x, dim=(-2, -1))
RuntimeError: Unsupported dtype BFloat16

* remove whitespace in torch_utils.py

* Update src/diffusers/utils/torch_utils.py

* Update torch_utils.py

---------

Co-authored-by: hlky <hlky@hlky.ac>
 2024-12-23 13:02:18 +05:30
Junsong Chen
1d9eb2c1d9 [DC-AE] Add the official Deep Compression Autoencoder code(32x,64x,128x compression ratio); (#9708) 
* first add a script for DC-AE;

* DC-AE init

* replace triton with custom implementation

* 1. rename file and remove un-used codes;

* no longer rely on omegaconf and dataclass

* replace custom activation with diffuers activation

* remove dc_ae attention in attention_processor.py

* iinherit from ModelMixin

* inherit from ConfigMixin

* dc-ae reduce to one file

* update downsample and upsample

* clean code

* support DecoderOutput

* remove get_same_padding and val2tuple

* remove autocast and some assert

* update ResBlock

* remove contents within super().__init__

* Update src/diffusers/models/autoencoders/dc_ae.py

Co-authored-by: YiYi Xu <yixu310@gmail.com>

* remove opsequential

* update other blocks to support the removal of build_norm

* remove build encoder/decoder project in/out

* remove inheritance of RMSNorm2d from LayerNorm

* remove reset_parameters for RMSNorm2d

Co-authored-by: YiYi Xu <yixu310@gmail.com>

* remove device and dtype in RMSNorm2d __init__

Co-authored-by: YiYi Xu <yixu310@gmail.com>

* Update src/diffusers/models/autoencoders/dc_ae.py

Co-authored-by: YiYi Xu <yixu310@gmail.com>

* Update src/diffusers/models/autoencoders/dc_ae.py

Co-authored-by: YiYi Xu <yixu310@gmail.com>

* Update src/diffusers/models/autoencoders/dc_ae.py

Co-authored-by: YiYi Xu <yixu310@gmail.com>

* remove op_list & build_block

* remove build_stage_main

* change file name to autoencoder_dc

* move LiteMLA to attention.py

* align with other vae decode output;

* add DC-AE into init files;

* update

* make quality && make style;

* quick push before dgx disappears again

* update

* make style

* update

* update

* fix

* refactor

* refactor

* refactor

* update

* possibly change to nn.Linear

* refactor

* make fix-copies

* replace vae with ae

* replace get_block_from_block_type to get_block

* replace downsample_block_type from Conv to conv for consistency

* add scaling factors

* incorporate changes for all checkpoints

* make style

* move mla to attention processor file; split qkv conv to linears

* refactor

* add tests

* from original file loader

* add docs

* add standard autoencoder methods

* combine attention processor

* fix tests

* update

* minor fix

* minor fix

* minor fix & in/out shortcut rename

* minor fix

* make style

* fix paper link

* update docs

* update single file loading

* make style

* remove single file loading support; todo for DN6

* Apply suggestions from code review

Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>

* add abstract

---------

Co-authored-by: Junyu Chen <chenjydl2003@gmail.com>
Co-authored-by: YiYi Xu <yixu310@gmail.com>
Co-authored-by: chenjy2003 <70215701+chenjy2003@users.noreply.github.com>
Co-authored-by: Aryan <aryan@huggingface.co>
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
 2024-12-23 13:02:18 +05:30
suzukimain
d54c70bcf6 [community] Load Models from Sources like Civitai into Existing Pipelines (#9986) 
* Added example of model search.

* Combine processing into one file

* Add parameters for base model

* Bug Fixes

* bug fix

* Create README.md

* Update search_for_civitai_and_HF.py

* Create requirements.txt

* bug fix

* Update README.md

* bug fix

* Correction of typos

* Update examples/model_search/README.md

Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>

* Update examples/model_search/README.md

Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>

* Update examples/model_search/README.md

Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>

* Update examples/model_search/README.md

Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>

* Update examples/model_search/README.md

Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>

* Update examples/model_search/README.md

Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>

* apply the changes

* Replace search_for_civitai_and_HF.py with pipeline_easy.py

* Update examples/model_search/README.md

Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>

* Update examples/model_search/README.md

Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>

* Update examples/model_search/README.md

Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>

* Update README.md

* Organize the table of parameters

* Update README.md

* Update README.md

* Update README.md

* make style

* Fixing the style of pipeline

* Fix pipeline style

* fix

---------

Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
 2024-12-23 13:02:18 +05:30
Aryan
b913ca1575 Remove duplicate checks for len(generator) != batch_size when generator is a list (#10134) 
remove duplicate checks
 2024-12-23 13:02:18 +05:30
Aritra Roy Gosthipaty
d0b901017a [Guide] Quantize your Diffusion Models with bnb (#10012) 
* chore: initial draft

* Apply suggestions from code review

Co-authored-by: Pedro Cuenca <pedro@huggingface.co>
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>

* chore: link in place

* chore: review suggestions

* Apply suggestions from code review

Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>

* chore: review suggestions

* Update docs/source/en/quantization/bitsandbytes.md

Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>

* review suggestions

* chore: review suggestions

* Apply suggestions from code review

Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>

* adding same changes to 4 bit section

* review suggestions

---------

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
Co-authored-by: Pedro Cuenca <pedro@huggingface.co>
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
 2024-12-23 13:02:18 +05:30
SahilCarterr
b49984081e [FIX] Bug in FluxPosEmbed (#10115) 
* Fix get_1d_rotary_pos_embed in embedding.py

* Update embeddings.py

---------

Co-authored-by: hlky <hlky@hlky.ac>
 2024-12-23 13:02:18 +05:30
Sayak Paul
9fcf353c80 [Tests] xfail incompatible SD configs. (#10127) 
* xfail incompatible SD configs.

* fix
 2024-12-23 13:02:18 +05:30
Sayak Paul
863dcc3b37 [Tests] fix condition argument in xfail. (#10099) 
* fix condition argument in xfail.

* revert init changes.
 2024-12-23 13:02:18 +05:30
Aryan
6bf8b2bf97 Use torch.device instead of current device index for BnB quantizer (#10069) 
* update

* apply review suggestion

---------

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
 2024-12-23 13:02:18 +05:30
Steven Liu
98413880d8 [docs] load_lora_adapter (#10119) 
* load_lora_adapter

* save

---------

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
 2024-12-23 13:02:18 +05:30
YiYi Xu
8fb2b6b5da pass attn mask arg for flux (#10122) 2024-12-23 13:02:18 +05:30
linjiapro
053a48dd43 Fix a bug for SD35 control net training and improve control net block index (#10065) 
* wip

---------

Co-authored-by: YiYi Xu <yixu310@gmail.com>
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
 2024-12-23 13:02:18 +05:30
zhangp365
ba0f59b47a Fix a bug in the state dict judgment in ip_adapter.py. (#10095) 
* fix a judging state dict bug in ip_adapter.py

* make

---------

Co-authored-by: hlky <hlky@hlky.ac>
 2024-12-23 13:02:18 +05:30
Linoy Tsaban
8dff843847 [Flux Redux] add prompt & multiple image input (#10056) 
* add multiple prompts to flux redux

---------

Co-authored-by: hlky <hlky@hlky.ac>
 2024-12-23 13:02:18 +05:30
hlky
488b2a62e7 Add sigmas to pipelines using FlowMatch (#10116) 2024-12-23 13:02:18 +05:30
Parag Ekbote
48d9c0908b Fix Broken Links in ReadMe (#10117) 
Update broken links in ReadME.
 2024-12-23 13:02:18 +05:30
Sayak Paul
57acfe7180 [bitsandbytes] allow directly CUDA placements of pipelines loaded with bnb components (#9840) 
* allow device placement when using bnb quantization.

* warning.

* tests

* fixes

* docs.

* require accelerate version.

* remove print.

* revert to()

* tests

* fixes

* fix: missing AutoencoderKL lora adapter (#9807)

* fix: missing AutoencoderKL lora adapter

* fix

---------

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>

* fixes

* fix condition test

* updates

* updates

* remove is_offloaded.

* fixes

* better

* empty

---------

Co-authored-by: Emmanuel Benazera <emmanuel.benazera@jolibrain.com>
 2024-12-23 13:02:18 +05:30
hlky
f80510b47a Fix pipeline_stable_audio formating (#10114) 2024-12-23 13:02:18 +05:30
fancy45daddy
de77e2236d add torch_xla support in pipeline_stable_audio.py (#10109) 
Update pipeline_stable_audio.py
 2024-12-23 13:02:18 +05:30
Sayak Paul
c71bdb70e5 [tests] refactor vae tests (#9808) 
* add: autoencoderkl tests

* autoencodertiny.

* fix

* asymmetric autoencoder.

* more

* integration tests for stable audio decoder.

* consistency decoder vae tests

* remove grad check from consistency decoder.

* cog

* bye test_models_vae.py

* fix

* fix

* remove allegro

* fixes

* fixes

* fixes

---------

Co-authored-by: Dhruv Nair <dhruv.nair@gmail.com>
 2024-12-23 13:02:18 +05:30
Ivan Skorokhodov
c33fd8b5ef Use parameters + buffers when deciding upscale_dtype (#9882) 
Sometimes, the decoder might lack parameters and only buffers (e.g., this happens when we manually need to convert all the parameters to buffers — e.g. to avoid packing fp16 and fp32 parameters with FSDP)
 2024-12-23 13:02:18 +05:30
hlky
e50b36f759 Test skip_guidance_layers in SD3 pipeline (#10102) 
* Test `skip_guidance_layers` in pipelines

* Move to test_pipeline_stable_diffusion_3
 2024-12-23 13:02:18 +05:30
hlky
2f68e52db5 Fix multi-prompt inference (#10103) 
* Fix multi-prompt inference

Fix generation of multiple images with multiple prompts, e.g len(prompts)>1, num_images_per_prompt>1

* make

* fix copies

---------

Co-authored-by: Nikita Balabin <nikita@mxl.ru>
 2024-12-23 13:02:18 +05:30
Anand Kumar
69acaeec4b [Bug fix] "previous_timestep()" in DDPM scheduling compatible with "trailing" and "linspace" options (#9384) 
* Update scheduling_ddpm.py

* fix copies

---------

Co-authored-by: YiYi Xu <yixu310@gmail.com>
Co-authored-by: hlky <hlky@hlky.ac>
 2024-12-23 13:02:18 +05:30
lsb
94ee4c1f0a Avoid compiling a progress bar. (#10098) 
* Avoid creating a progress bar when it is disabled.

This is useful when exporting a pipeline, and allows a compiler to avoid trying to compile away tqdm.

* Prevent the PyTorch compiler from compiling progress bars.

* Update pipeline_utils.py
 2024-12-23 13:02:18 +05:30
aihao
70557b92f2 update (#7067) 
* add data_dir parameter to load_dataset

---------

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
Co-authored-by: YiYi Xu <yixu310@gmail.com>
Co-authored-by: hlky <hlky@hlky.ac>
 2024-12-23 13:02:18 +05:30
StAlKeR7779
af7efa0454 DPM++ third order fixes (#9104) 
* Fix wrong output on 3n-1 steps count

* Add sde handling to 3 order

* make

* copies

---------

Co-authored-by: hlky <hlky@hlky.ac>
 2024-12-23 13:02:18 +05:30
Parag Ekbote
a9f8f2ea87 Fix Broken Link in Optimization Docs (#10105) 
Update broken link.
 2024-12-23 13:02:18 +05:30
Parag Ekbote
d65cb943c3 Notebooks for Community Scripts-4 (#10094) 
* Add Diffuser Notebooks for Community Scripts.

* Add missing link.

* Styling Improvement.
 2024-12-23 13:02:18 +05:30
Benjamin Paine
4c6335c7b1 Add StableDiffusion3PAGImg2Img Pipeline + Fix SD3 Unconditional PAG (#9932) 
* fix progress bar updates in SD 1.5 PAG Img2Img pipeline



---------

Co-authored-by: Vinh H. Pham <phamvinh257@gmail.com>
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
 2024-12-23 13:02:18 +05:30
Pedro Cuenca
918aa10d21 Don't stale close-to-merge (#10096) 
Re: https://github.com/huggingface/diffusers/discussions/10046#discussioncomment-11443466
 2024-12-23 13:02:18 +05:30
DTG
598c25253b Fix some documentation in ./src/diffusers/models/embeddings.py for demo (#9579) 
* Fix some documentation in ./src/diffusers/models/embeddings.py as demonstration.


---------

Co-authored-by: DaAccursed05 <68813178+DaAccursed05@users.noreply.github.com>
Co-authored-by: Aryan <contact.aryanvs@gmail.com>
Co-authored-by: Aryan <aryan@huggingface.co>
Co-authored-by: YiYi Xu <yixu310@gmail.com>
 2024-12-23 13:02:18 +05:30
Lucain
5241ae4d86 Let server decide default repo visibility (#10047) 2024-12-23 13:02:18 +05:30
Emmanuel Benazera
3dd8aa4e30 fix: missing AutoencoderKL lora adapter (#9807) 
* fix: missing AutoencoderKL lora adapter

* fix

---------

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
 2024-12-23 13:02:18 +05:30
hlky
becef297a5 Convert sigmas to np.array in FlowMatch set_timesteps (#10088) 2024-12-23 13:02:18 +05:30