Commit Graph

4913 Commits

Author SHA1 Message Date
Sayak Paul
86e62ae4c5 change to 2024 in the license (#6902)
change to 2024
2024-12-23 13:02:07 +05:30
Masamune Ishihara
bd27402104 Add fps argument to export_to_gif function. (#6786) 2024-12-23 13:02:06 +05:30
Patryk Bartkowiak
ad7d79be5e changed positional parameters to named parameters like in docs (#6905)
Co-authored-by: Patryk Bartkowiak <patryk.bartkowiak@tcl.com>
Co-authored-by: Linoy Tsaban <57615435+linoytsaban@users.noreply.github.com>
2024-12-23 13:02:06 +05:30
Ehsan Akhgari
d15a0ecaaf Fix bug when converting checkpoint to diffusers format (#6900)
This fixes #6899.
2024-12-23 13:02:06 +05:30
Sayak Paul
0f2e44cd4d [I2VGenXL] attention_head_dim in the UNet (#6872)
* attention_head_dim

* debug

* print more info

* correct num_attention_heads behaviour

* down_block_num_attention_heads -> num_attention_heads.

* correct the image link in doc.

* add: deprecation for num_attention_head

* fix: test argument to use attention_head_dim

* more fixes.

* quality

* address comments.

* remove depcrecation.
2024-12-23 13:02:06 +05:30
Sayak Paul
a22cc563ee [IP Adapters] introduce ip_adapter_image_embeds in the SD pipeline call (#6868)
* add: support for passing ip adapter image embeddings

* debugging

* make feature_extractor unloading conditioned on safety_checker

* better condition

* type annotation

* index to look into value slices

* more debugging

* debugging

* serialize embeddings dict

* better conditioning

* remove unnecessary prints.

* Update src/diffusers/loaders/ip_adapter.py

Co-authored-by: YiYi Xu <yixu310@gmail.com>

* make fix-copies and styling.

* styling and further copy fixing.

* fix: check_inputs call in controlnet sdxl img2img pipeline

---------

Co-authored-by: YiYi Xu <yixu310@gmail.com>
2024-12-23 13:02:06 +05:30
Srimanth Agastyaraju
e37350a03a Fix: training resume from fp16 for SDXL Consistency Distillation (#6840)
* Fix: training resume from fp16 for lcm distill lora sdxl

* Fix coding quality - run linter

* Fix 1 - shift mixed precision cast before optimizer

* Fix 2 - State dict errors by removing load_lora_into_unet

* Update train_lcm_distill_lora_sdxl.py - Revert default cache dir to None

---------

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
2024-12-23 13:02:06 +05:30
Sayak Paul
d08deb7d52 Remove torch_dtype in to() to end deprecation (#6886)
* remove torch_dtype from to()

* remove torch_dtype from usage scripts.

* remove old lora backend

* Revert "remove old lora backend"

This reverts commit adcddf6ba4.
2024-12-23 13:02:06 +05:30
camaro
14aaf03f7d fix: keyword argument mismatch (#6895) 2024-12-23 13:02:06 +05:30
YiYi Xu
81f60afbfe [ip-adapter] make sure length of scale is same as number of ip-adapters when using set_ip_adapter_scale (#6884)
add

Co-authored-by: yiyixuxu <yixu310@gmail,com>
2024-12-23 13:02:06 +05:30
Sayak Paul
3a06d3a15e [Model Card] standardize dreambooth model card (#6729)
* feat: standarize model card creation for dreambooth training.

* correct 'inference

* remove comments.

* take component out of kwargs

* style

* add: card template to have a leaner description.

* widget support.

* propagate changes to train_dreambooth_lora

* propagate changes to custom diffusion

* make widget properly type-annotated
2024-12-23 13:02:06 +05:30
Félix Sanz
6d0fe4ea94 fix: typo in callback function name and property (#6834)
* fix: callback function name is incorrect

On this tutorial there is a function defined and then used inside `callback_on_step_end` argument, but the name was not correct (mismatch)

* fix: typo in num_timestep (correct is num_timesteps)

fixed property name
2024-12-23 13:02:06 +05:30
Dhruv Nair
0b0a2b8692 Fix last IP Adapter test (#6875)
update
2024-12-23 13:02:06 +05:30
Dhruv Nair
d059c40ce5 Refactor Deepfloyd IF tests. (#6855)
* update

* update

* update
2024-12-23 13:02:06 +05:30
sayakpaul
444b1c6d19 Revert "add attention_head_dim"
This reverts commit 15f6b22466.
2024-12-23 13:02:06 +05:30
sayakpaul
45bfd204b4 add attention_head_dim 2024-12-23 13:02:06 +05:30
Sayak Paul
23356d176c [I2vGenXL] clean up things (#6845)
* remove _to_tensor

* remove _to_tensor definition

* remove _collapse_frames_into_batch

* remove lora for not bloating the code.

* remove sample_size.

* simplify code a bit more

* ensure timesteps are always in tensor.
2024-12-23 13:02:06 +05:30
Edward Li
1de937ed2f Fix AutoencoderTiny with use_slicing (#6850)
* Fix `AutoencoderTiny` with `use_slicing`

When using slicing with AutoencoderTiny, the encoder mistakenly encodes the entire batch for every image in the batch.

* Fixed formatting issue
2024-12-23 13:02:06 +05:30
Dhruv Nair
f315811aed Add single file and IP Adapter support to PIA Pipeline (#6851)
update
2024-12-23 13:02:06 +05:30
Dhruv Nair
216d3457b2 Update IP Adapter tests to use cosine similarity distance (#6806)
* update

* update
2024-12-23 13:02:06 +05:30
Dhruv Nair
3f3e2fad98 Fix posix path issue in testing utils (#6849)
update
2024-12-23 13:02:06 +05:30
小咩Goat
20e756351b Fix forward pass in UNetMotionModel when gradient checkpoint is enabled (#6744)
fix #6742

Co-authored-by: Dhruv Nair <dhruv.nair@gmail.com>
2024-12-23 13:02:06 +05:30
YiYi Xu
5c90243c0d update IP-adapter code in UNetMotionModel (#6828)
fix

Co-authored-by: yiyixuxu <yixu310@gmail,com>
2024-12-23 13:02:06 +05:30
Dhruv Nair
d44a09475d Multiple small fixes to Video Pipeline docs (#6805)
* update

* update

* update

* Update src/diffusers/pipelines/i2vgen_xl/pipeline_i2vgen_xl.py

Co-authored-by: YiYi Xu <yixu310@gmail.com>

* update

* update

---------

Co-authored-by: YiYi Xu <yixu310@gmail.com>
2024-12-23 13:02:06 +05:30
YiYi Xu
03aa6bb279 add self.use_ada_layer_norm_* params back to BasicTransformerBlock (#6841)
fix sd reference community ppeline

Co-authored-by: yiyixuxu <yixu310@gmail,com>
2024-12-23 13:02:06 +05:30
Linoy Tsaban
7f5335214a [dreambooth lora sdxl] add sdxl micro conditioning (#6795)
* add micro conditioning

* remove redundant lines

* style

* fix missing 's'

* fix missing shape bug due to missing RGB if statement

* remove redundant if, change arg order

---------

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
2024-12-23 13:02:06 +05:30
Fabio Rigano
0e39ac0f41 Bugfix in IPAdapterFaceID (#6835) 2024-12-23 13:02:06 +05:30
Linoy Tsaban
75c5710669 [advanced dreambooth lora sdxl script] new features + bug fixes (#6691)
* add noise_offset param

* micro conditioning - wip

* image processing adjusted and moved to support micro conditioning

* change time ids to be computed inside train loop

* change time ids to be computed inside train loop

* change time ids to be computed inside train loop

* time ids shape fix

* move token replacement of validation prompt to the same section of instance prompt and class prompt

* add offset noise to sd15 advanced script

* fix token loading during validation

* fix token loading during validation in sdxl script

* a little clean

* style

* a little clean

* style

* sdxl script - a little clean + minor path fix

sd 1.5 script - change default resolution value

* ad 1.5 script - minor path fix

* fix missing comma in code example in model card

* clean up commented lines

* style

* remove time ids computed outside training loop - no longer used now that we utilize micro-conditioning, as all time ids are now computed inside the training loop

* style

* [WIP] - added draft readme, building off of examples/dreambooth/README.md

* readme

* readme

* readme

* readme

* readme

* readme

* readme

* readme

* removed --crops_coords_top_left from CLI args

* style

* fix missing shape bug due to missing RGB if statement

* add blog mention at the start of the reamde as well

* Update examples/advanced_diffusion_training/README.md

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>

* change note to render nicely as well

---------

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
2024-12-23 13:02:06 +05:30
Stephen
d167dd19b4 Change path to posix (testing_utils.py) (#6803)
change path to pathlib as_posix

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
2024-12-23 13:02:06 +05:30
Younes Belkada
af99e2ec1b Fixes LoRA SDXL training script with DDP + PEFT (#6816)
Update train_dreambooth_lora_sdxl.py
2024-12-23 13:02:06 +05:30
UmerHA
f44425831e [Contributor Experience] Fix test collection on MPS (#6808)
* Update testing_utils.py

* Update testing_utils.py
2024-12-23 13:02:06 +05:30
YiYi Xu
c400e58b05 [refactor]Scheduler.set_begin_index (#6728) 2024-12-23 13:02:06 +05:30
Sayak Paul
1ae08ed607 [Refactor] harmonize the module structure for models in tests (#6738)
* harmonize the module structure for models in tests

* make the folders modules.

---------

Co-authored-by: YiYi Xu <yixu310@gmail.com>
2024-12-23 13:02:06 +05:30
YiYi Xu
3458cc43f4 add is_torchvision_available (#6800)
* add

* remove transformer

---------

Co-authored-by: yiyixuxu <yixu310@gmail,com>
2024-12-23 13:02:06 +05:30
Patrick von Platen
eab6cc6f37 fix torchvision import (#6796) 2024-12-23 13:02:06 +05:30
Sayak Paul
804d50ec24 [Feat] add I2VGenXL for image-to-video generation (#6665)
---------

Co-authored-by: Dhruv Nair <dhruv.nair@gmail.com>
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
Co-authored-by: YiYi Xu <yixu310@gmail.com>
2024-12-23 13:02:06 +05:30
YiYi Xu
388d8c0d8e [sdxl k-diffusion pipeline]move sigma to device (#6757)
move sigma to device

Co-authored-by: yiyixuxu <yixu310@gmail,com>
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
2024-12-23 13:02:06 +05:30
YiYi Xu
f1e105e664 [IP-Adapter] Support multiple IP-Adapters (#6573)
---------

Co-authored-by: yiyixuxu <yixu310@gmail,com>
Co-authored-by: Alvaro Somoza <somoza.alvaro@gmail.com>
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
2024-12-23 13:02:06 +05:30
Steven Liu
7893411aed [docs] Add missing parameter (#6775)
add missing param
2024-12-23 13:02:06 +05:30
Dhruv Nair
a31ebdc56c Add PIA Model/Pipeline (#6698)
* update

* update

* updaet

* add tests and docs

* clean up

* add to toctree

* fix copies

* pr review feedback

* fix copies

* fix tests

* update docs

* update

* update

* update docs

* update

* update

* update

* update
2024-12-23 13:02:06 +05:30
Dhruv Nair
6442007487 Fix setting scaling factor in VAE config (#6779)
fix
2024-12-23 13:02:06 +05:30
Kashif Rasul
6a7cfa2a20 add ipo, hinge and cpo loss to dpo trainer (#6788)
add ipo and hinge loss to dpo trainer
2024-12-23 13:02:06 +05:30
Sayak Paul
9b19693ed0 [Kandinsky tests] add is_flaky to test_model_cpu_offload_forward_pass (#6762)
* add is_flaky to test_model_cpu_offload_forward_pass

* style

* update

---------

Co-authored-by: Dhruv Nair <dhruv.nair@gmail.com>
2024-12-23 13:02:06 +05:30
Dhruv Nair
28af1db5b9 Pin torch < 2.2.0 in test runners (#6780)
* update

* update

---------

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
2024-12-23 13:02:06 +05:30
Viet Nguyen
7d2cbd8576 Update train_diffusion_dpo.py (#6754)
* Update train_diffusion_dpo.py

Address #6702

* Update train_diffusion_dpo_sdxl.py

* Empty-Commit

---------

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
2024-12-23 13:02:06 +05:30
Sayak Paul
4485d34f2f add note about serialization (#6764) 2024-12-23 13:02:06 +05:30
dg845
48ae606e50 Fix bug in ResnetBlock2D.forward where LoRA Scale gets Overwritten (#6736)
Fix bug in ResnetBlock2D.forward when not USE_PEFT_BACKEND and using scale_shift for time emb where the lora scale  gets overwritten.

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
2024-12-23 13:02:06 +05:30
Yunxuan Xiao
4a0378e26b [DDPMScheduler] Load alpha_cumprod to device to avoid redundant data movement. (#6704)
* load cumprod tensor to device

Signed-off-by: woshiyyya <xiaoyunxuan1998@gmail.com>

* fixing ci

Signed-off-by: woshiyyya <xiaoyunxuan1998@gmail.com>

* make fix-copies

Signed-off-by: woshiyyya <xiaoyunxuan1998@gmail.com>

---------

Signed-off-by: woshiyyya <xiaoyunxuan1998@gmail.com>
2024-12-23 13:02:06 +05:30
Dhruv Nair
75a630cc64 Update export to video to support new tensor_to_vid function in video pipelines (#6715)
update
2024-12-23 13:02:06 +05:30
YiYi Xu
302aa3e67a udpate ip-adapter slow tests (#6760)
* udpate slices

* up

* hopefully last one

---------

Co-authored-by: yiyixuxu <yixu310@gmail,com>
2024-12-23 13:02:06 +05:30