LLM/diffusers
1
0
Fork 0
mirror of https://github.com/huggingface/diffusers.git synced 2026-02-13 22:35:37 +08:00
Code Issues Packages Projects Releases Wiki Activity
4,913 Commits 744 Branches 94 Tags
4c5082ca685d1dd0a24907890da170e258e5cd01
Commit Graph

4913 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
sayakpaul
4c5082ca68 updates 2024-12-23 13:02:19 +05:30
Sayak Paul
fb17ba3b9d [SANA LoRA] sana lora training tests and misc. (#10296) 
* sana lora training tests and misc.

* remove push to hub

* Update examples/dreambooth/train_dreambooth_lora_sana.py

Co-authored-by: Aryan <aryan@huggingface.co>

---------

Co-authored-by: Aryan <aryan@huggingface.co>
 2024-12-23 13:02:19 +05:30
Aryan
1d6c9f4e8b [tests] Refactor TorchAO serialization fast tests (#10271) 
refactor
 2024-12-23 13:02:19 +05:30
Sayak Paul
2ff65127be [docs] fix: torchao example. (#10278) 
fix: torchao example.
 2024-12-23 13:02:19 +05:30
Aryan
01acf5f42a Bump minimum TorchAO version to 0.7.0 (#10293) 
* bump min torchao version to 0.7.0

* update
 2024-12-23 13:02:19 +05:30
Thien Tran
2bc919fe87 .from_single_file() - Add missing .shape (#10332) 
Add missing `.shape`
 2024-12-23 13:02:19 +05:30
Junsong Chen
e6e3bd944a [Sana bug] bug fix for 2K model config (#10340) 
* fix the Positinoal Embedding bug in 2K model;

* Change the default model to the BF16 one for more stable training and output

* make style

* substract buffer size

* add compute_module_persistent_sizes

---------

Co-authored-by: yiyixuxu <yixu310@gmail.com>
 2024-12-23 13:02:19 +05:30
Dhruv Nair
a6f7931320 [Single File] Add Single File support for HunYuan video (#10320) 
* update

* Update src/diffusers/loaders/single_file_utils.py

Co-authored-by: Aryan <aryan@huggingface.co>

---------

Co-authored-by: Aryan <aryan@huggingface.co>
 2024-12-23 13:02:19 +05:30
YiYi Xu
bbc2ffce52 update get_parameter_dtype (#10342) 
add:
q
 2024-12-23 13:02:19 +05:30
hlky
2374befc80 Fix enable_sequential_cpu_offload in test_kandinsky_combined (#10324) 
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
 2024-12-23 13:02:19 +05:30
Aryan
2b082b6e4e Community hosted weights for diffusers format HunyuanVideo weights (#10344) 
update docs and example to use community weights
 2024-12-23 13:02:19 +05:30
Mehmet Yiğit Özgenç
c40a5590a4 flux controlnet inpaint config bug (#10291) 
* flux controlnet inpaint config bug

* Update src/diffusers/pipelines/flux/pipeline_flux_controlnet_inpainting.py

---------

Co-authored-by: yigitozgenc <yigit@quantuslabs.ai>
Co-authored-by: hlky <hlky@hlky.ac>
 2024-12-23 13:02:19 +05:30
hlky
ebd079ad91 Support Flux IP Adapter (#10261) 
* Flux IP-Adapter

* test cfg

* make style

* temp remove copied from

* fix test

* fix test

* v2

* fix

* make style

* temp remove copied from

* Apply suggestions from code review

Co-authored-by: YiYi Xu <yixu310@gmail.com>

* Move encoder_hid_proj to inside FluxTransformer2DModel

* merge

* separate encode_prompt, add copied from, image_encoder offload

* make

* fix test

* fix

* Update src/diffusers/pipelines/flux/pipeline_flux.py

* test_flux_prompt_embeds change not needed

* true_cfg -> true_cfg_scale

* fix merge conflict

* test_flux_ip_adapter_inference

* add fast test

* FluxIPAdapterMixin not test mixin

* Update pipeline_flux.py

Co-authored-by: YiYi Xu <yixu310@gmail.com>

---------

Co-authored-by: YiYi Xu <yixu310@gmail.com>
 2024-12-23 13:02:19 +05:30
hlky
e4701c183d Fix EMAModel test_from_pretrained (#10325) 2024-12-23 13:02:19 +05:30
hlky
824ebbfb1e Fix push_tests_mps.yml (#10326) 2024-12-23 13:02:19 +05:30
Sayak Paul
6403f9446a [Docs] Update gguf.md to remove generator from the pipeline from_pretrained (#10299) 
Update gguf.md to remove generator from the pipeline from_pretrained
 2024-12-23 13:02:19 +05:30
Junsong Chen
678582afc6 [Sana]add 2K related model for Sana (#10322) 
add 2K related model for Sana
 2024-12-23 13:02:19 +05:30
Steven Liu
bf312863cc [docs] Fix quantization links (#10323) 
Update overview.md
 2024-12-23 13:02:19 +05:30
Aditya Raj
1964267573 [BUG FIX] [Stable Audio Pipeline] Resolve torch.Tensor.new_zeros() TypeError in function prepare_latents caused by audio_vae_length (#10306) 
[BUG FIX] [Stable Audio Pipeline] TypeError: new_zeros(): argument 'size' failed to unpack the object at pos 3 with error "type must be tuple of ints,but got float"

torch.Tensor.new_zeros() takes a single argument size (int...) – a list, tuple, or torch.Size of integers defining the shape of the output tensor.

in function prepare_latents:
audio_vae_length = self.transformer.config.sample_size * self.vae.hop_length
audio_shape = (batch_size // num_waveforms_per_prompt, audio_channels, audio_vae_length)
...
audio = initial_audio_waveforms.new_zeros(audio_shape)

audio_vae_length evaluates to float because self.transformer.config.sample_size returns a float

Co-authored-by: hlky <hlky@hlky.ac>
 2024-12-23 13:02:19 +05:30
Leojc
47918a87d3 docs: fix a mistake in docstring (#10319) 
Update pipeline_hunyuan_video.py

docs: fix a mistake
 2024-12-23 13:02:19 +05:30
Sayak Paul
2f26efb2d5 [Docs] Update ltx_video.md to remove generator from from_pretrained() (#10316) 
Update ltx_video.md to remove generator from `from_pretrained()`
 2024-12-23 13:02:19 +05:30
Dhruv Nair
87e76b6ab4 Mochi docs (#9934) 
* update

* update

* update

* update

* update

---------

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
 2024-12-23 13:02:19 +05:30
Sayak Paul
c80f7f1569 [Tests] add integration tests for lora expansion stuff in Flux. (#10318) 
add integration tests for lora expansion stuff in Flux.
 2024-12-23 13:02:19 +05:30
Sayak Paul
5abff4e200 [LoRA] feat: support loading regular Flux LoRAs into Flux Control, and Fill (#10259) 
* lora expansion with dummy zeros.

* updates

* fix working 🥳

* working.

* use torch.device meta for state dict expansion.

* tests

Co-authored-by: a-r-r-o-w <contact.aryanvs@gmail.com>

* fixes

* fixes

* switch to debug

* fix

* Apply suggestions from code review

Co-authored-by: Aryan <aryan@huggingface.co>

* fix stuff

* docs

---------

Co-authored-by: a-r-r-o-w <contact.aryanvs@gmail.com>
Co-authored-by: Aryan <aryan@huggingface.co>
 2024-12-23 13:02:19 +05:30
Dhruv Nair
bb482bac26 [Single File] Add GGUF support for LTX (#10298) 
* update

* add docs.

---------

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
 2024-12-23 13:02:19 +05:30
Aryan
7ffa043cb8 Make tensors in ResNet contiguous for Hunyuan VAE (#10309) 
contiguous tensors in resnet

Co-authored-by: YiYi Xu <yixu310@gmail.com>
 2024-12-23 13:02:19 +05:30
Aryan
971d334e2b Add support for sharded models when TorchAO quantization is enabled (#10256) 
* add sharded + device_map check
 2024-12-23 13:02:19 +05:30
Daniel Regado
b571adfb40 [WIP] SD3.5 IP-Adapter Pipeline Integration (#9987) 
* Added support for single IPAdapter on SD3.5 pipeline



---------

Co-authored-by: hlky <hlky@hlky.ac>
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
Co-authored-by: YiYi Xu <yixu310@gmail.com>
 2024-12-23 13:02:19 +05:30
dg845
27baf7b086 Enable Gradient Checkpointing for UNet2DModel (New) (#7201) 
* Port UNet2DModel gradient checkpointing code from #6718.


---------

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
Co-authored-by: Vincent Neemie <92559302+VincentNeemie@users.noreply.github.com>
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
Co-authored-by: Dhruv Nair <dhruv.nair@gmail.com>
Co-authored-by: hlky <hlky@hlky.ac>
 2024-12-23 13:02:19 +05:30
djm
a8a4e18064 unet's sample_size attribute is to accept tuple(h, w) in StableDiffusionPipeline (#10181) 2024-12-23 13:02:19 +05:30
Aryan
4a18b3e11f Fix failing lora tests after HunyuanVideo lora (#10307) 
fix
 2024-12-23 13:02:19 +05:30
hlky
46f03a6479 Fix local_files_only for checkpoints with shards (#10294) 2024-12-23 13:02:19 +05:30
Dhruv Nair
684d296fbc Allow Mochi Transformer to be split across multiple GPUs (#10300) 
update
 2024-12-23 13:02:19 +05:30
Dhruv Nair
26d7b8494a [Single File] Add single file support for Mochi Transformer (#10268) 
update
 2024-12-23 13:02:19 +05:30
Shenghai Yuan
e854770275 [LoRA] Support HunyuanVideo (#10254) 
* 1217

* 1217

* 1217

* update

* reverse

* add test

* update test

* make style

* update

* make style

---------

Co-authored-by: Aryan <aryan@huggingface.co>
 2024-12-23 13:02:19 +05:30
hlky
49ce471974 Check correct model type is passed to from_pretrained (#10189) 
* Check correct model type is passed to `from_pretrained`

* Flax, skip scheduler

* test_wrong_model

* Fix for scheduler

* Update tests/pipelines/test_pipelines.py

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>

* EnumMeta

* Flax

* scheduler in expected types

* make

* type object 'CLIPTokenizer' has no attribute '_PipelineFastTests__name'

* support union

* fix typing in kandinsky

* make

* add LCMScheduler

* 'LCMScheduler' object has no attribute 'sigmas'

* tests for wrong scheduler

* make

* update

* warning

* tests

* Update src/diffusers/pipelines/pipeline_utils.py

Co-authored-by: Dhruv Nair <dhruv.nair@gmail.com>

* import FlaxSchedulerMixin

* skip scheduler

---------

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
Co-authored-by: Dhruv Nair <dhruv.nair@gmail.com>
 2024-12-23 13:02:19 +05:30
赵三石
84fe2e1a44 Update lora_conversion_utils.py (#9980) 
x-flux single-blocks lora load

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
Co-authored-by: YiYi Xu <yixu310@gmail.com>
 2024-12-23 13:02:19 +05:30
hlky
8f3e45f52c Add Flux Control to AutoPipeline (#10292) 2024-12-23 13:02:19 +05:30
Aryan
6d7896e782 Hunyuan VAE tiling fixes and transformer docs (#10295) 
* update

* udpate

* fix test
 2024-12-23 13:02:19 +05:30
Sayak Paul
99885f3c55 [chore] Update README_sana.md to update the default model (#10285) 
Update README_sana.md to update the default model
 2024-12-23 13:02:19 +05:30
Aryan
c1a7c92bf9 [tests] remove nullop import checks from lora tests (#10273) 
remove nullop imports
 2024-12-23 13:02:19 +05:30
Aryan
a9c761f047 Rename Mochi integration test correctly (#10220) 
rename integration test
 2024-12-23 13:02:19 +05:30
Aryan
64e4e0a378 [tests] Fix broken cuda, nightly and lora tests on main for CogVideoX (#10270) 
fix joint pos embedding device
 2024-12-23 13:02:19 +05:30
Dhruv Nair
c73d1a0d62 [Single File] Add single file support for Flux Canny, Depth and Fill (#10288) 
update
 2024-12-23 13:02:19 +05:30
Dhruv Nair
b917a719d7 Make zeroing prompt embeds for Mochi Pipeline configurable (#10284) 
update
 2024-12-23 13:02:19 +05:30
Dhruv Nair
b723a81be7 Fix Doc links in GGUF and Quantization overview docs (#10279) 
* update

* Update docs/source/en/quantization/gguf.md

Co-authored-by: Aryan <aryan@huggingface.co>

---------

Co-authored-by: Aryan <aryan@huggingface.co>
 2024-12-23 13:02:19 +05:30
hlky
f0f2091bf3 Fix sigma_last with use_flow_sigmas (#10267) 2024-12-23 13:02:19 +05:30
Andrés Romero
7c78d05323 Flux Control(Depth/Canny) + Inpaint (#10192) 
* flux_control_inpaint - failing test_flux_different_prompts

* removing test_flux_different_prompts?

* fix style

* fix from PR comments

* fix style

* reducing guidance_scale in demo

* Update src/diffusers/pipelines/flux/pipeline_flux_control_inpaint.py

Co-authored-by: hlky <hlky@hlky.ac>

* make

* prepare_latents is not copied from

* update docs

* typos

---------

Co-authored-by: affromero <ubuntu@ip-172-31-17-146.ec2.internal>
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
Co-authored-by: hlky <hlky@hlky.ac>
 2024-12-23 13:02:19 +05:30
Qin Zhou
c7c5024048 Support pass kwargs to sd3 custom attention processor (#9818) 
* Support pass kwargs to sd3 custom attention processor


---------

Co-authored-by: hlky <hlky@hlky.ac>
Co-authored-by: YiYi Xu <yixu310@gmail.com>
 2024-12-23 13:02:19 +05:30
Xinyuan Zhao
8cee828cbe Make time_embed_dim of UNet2DModel changeable (#10262) 2024-12-23 13:02:19 +05:30