LLM/diffusers
1
0
Fork 0
mirror of https://github.com/huggingface/diffusers.git synced 2026-02-14 14:55:26 +08:00
Code Issues Packages Projects Releases Wiki Activity
4,913 Commits 743 Branches 94 Tags
4c5082ca685d1dd0a24907890da170e258e5cd01
Commit Graph

4913 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
captainzz
c4e77f1ee1 fix bugs for sd3 controlnet training (#9489) 
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
Co-authored-by: YiYi Xu <yixu310@gmail.com>
 2024-12-23 13:02:16 +05:30
hlky
74c74629b3 Add Noise Schedule/Schedule Type to Schedulers Overview documentation (#9504) 
* Add Noise Schedule/Schedule Type to Schedulers Overview docs

* Update docs/source/en/api/schedulers/overview.md

Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>

---------

Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
 2024-12-23 13:02:16 +05:30
Seongbin Lim
078286aa85 Allow DDPMPipeline half precision (#9222) 
Co-authored-by: YiYi Xu <yixu310@gmail.com>
 2024-12-23 13:02:16 +05:30
hlky
7d92452bf9 [Schedulers] Add exponential sigmas / exponential noise schedule (#9499) 
* exponential sigmas

* Apply suggestions from code review

Co-authored-by: YiYi Xu <yixu310@gmail.com>

* make style

---------

Co-authored-by: YiYi Xu <yixu310@gmail.com>
 2024-12-23 13:02:16 +05:30
pibbo88
715a6c99a2 Fix the bug of sd3 controlnet training when using gradient checkpointing. (#9498) 
Fix the bug of sd3 controlnet training when using gradient_checkpointing. Refer to issue #9496
 2024-12-23 13:02:16 +05:30
M Saqlain
298963abbe [Tests] Reduce the model size in the lumina test (#8985) 
* Reduced model size for lumina-tests

* Handled failing tests
 2024-12-23 13:02:16 +05:30
Sayak Paul
8fb47719db [Cog] some minor fixes and nits (#9466) 
* fix positional arguments in check_inputs().

* add video and latetns to check_inputs().

* prep latents_in_channels.

* quality

* multiple fixes.

* fix
 2024-12-23 13:02:16 +05:30
Sayak Paul
53786f5eab [CI] fix nightly model tests (#9483) 
* check if default attn procs fix it.

* print

* print

* replace

* style./

* replace revision with variant.

* replace with stable-diffusion-v1-5/stable-diffusion-inpainting.

* replace with stable-diffusion-v1-5/stable-diffusion-v1-5.

* fix
 2024-12-23 13:02:16 +05:30
Aryan
91aa68b360 [refactor] LoRA tests (#9481) 
* refactor scheduler class usage

* reorder to make tests more readable

* remove pipeline specific checks and skip tests directly

* rewrite denoiser conditions cleaner

* bump tolerance for cog test
 2024-12-23 13:02:16 +05:30
Vladimir Mandic
e1ea942b42 Several fixes to Flux ControlNet pipelines (#9472) 
* fix flux controlnet pipelines

---------

Co-authored-by: yiyixuxu <yixu310@gmail.com>
 2024-12-23 13:02:16 +05:30
Aryan
fd15a9d7bd [training] CogVideoX Lora (#9302) 
* cogvideox lora training draft

* update

* update

* update

* update

* update

* make fix-copies

* update

* update

* apply suggestions from review

* apply suggestions from reveiw

* fix typo

* Update examples/cogvideo/train_cogvideox_lora.py

Co-authored-by: YiYi Xu <yixu310@gmail.com>

* fix lora alpha

* use correct lora scaling for final test pipeline

* Update examples/cogvideo/train_cogvideox_lora.py

Co-authored-by: YiYi Xu <yixu310@gmail.com>

* apply suggestions from review; prodigy optimizer

YiYi Xu <yixu310@gmail.com>

* add tests

* make style

* add README

* update

* update

* make style

* fix

* update

* add test skeleton

* revert lora utils changes

* add cleaner modifications to lora testing utils

* update lora tests

* deepspeed stuff

* add requirements.txt

* deepspeed refactor

* add lora stuff to img2vid pipeline to fix tests

* fight tests

* add co-authors

Co-Authored-By: Fu-Yun Wang <1697256461@qq.com>

Co-Authored-By: zR <2448370773@qq.com>

* fight lora runner tests

* import Dummy optim and scheduler only wheh required

* update docs

* add coauthors

Co-Authored-By: Fu-Yun Wang <1697256461@qq.com>

* remove option to train text encoder

Co-Authored-By: bghira <bghira@users.github.com>

* update tests

* fight more tests

* update

* fix vid2vid

* fix typo

* remove lora tests; todo in follow-up PR

* undo img2vid changes

* remove text encoder related changes in lora loader mixin

* Revert "remove text encoder related changes in lora loader mixin"

This reverts commit f8a8444487.

* update

* round 1 of fighting tests

* round 2 of fighting tests

* fix copied from comment

* fix typo in lora test

* update styling

Co-Authored-By: YiYi Xu <yixu310@gmail.com>

---------

Co-authored-by: YiYi Xu <yixu310@gmail.com>
Co-authored-by: zR <2448370773@qq.com>
Co-authored-by: Fu-Yun Wang <1697256461@qq.com>
Co-authored-by: bghira <bghira@users.github.com>
 2024-12-23 13:02:16 +05:30
Sayak Paul
240f753324 [Flux] add lora integration tests. (#9353) 
* add lora integration tests.

* internal note

* add a skip marker.
 2024-12-23 13:02:16 +05:30
Anatoly Belikov
bef6cfa785 adapt masked im2im pipeline for SDXL (#7790) 
* adapt masked im2im pipeline for SDXL

* usage for masked im2im stable diffusion XL pipeline

* style

* style

* style

---------

Co-authored-by: YiYi Xu <yixu310@gmail.com>
 2024-12-23 13:02:16 +05:30
Aryan
cb3a0defb9 set max_shard_size to None for pipeline save_pretrained (#9447) 
* update default max_shard_size

* add None check to fix tests

---------

Co-authored-by: YiYi Xu <yixu310@gmail.com>
 2024-12-23 13:02:16 +05:30
Aryan
0494cbd01a Remove CogVideoX mentions from single file docs; Test updates (#9444) 
* remove mentions from single file

* update tests

* update
 2024-12-23 13:02:16 +05:30
Subho Ghosh
4c08316cf7 Feature flux controlnet img2img and inpaint pipeline (#9408) 
* Implemented FLUX controlnet support to Img2Img pipeline
 2024-12-23 13:02:16 +05:30
Linoy Tsaban
502191f3a6 [Flux with CFG] add flux pipeline with cfg support (#9445) 
* true_cfg

* add check negative prompt/embeds inputs

* move to community pipelines

* move to community pipelines

* revert true cfg changes to the orig pipline

* style

---------

Co-authored-by: YiYi Xu <yixu310@gmail.com>
 2024-12-23 13:02:16 +05:30
Sayak Paul
043517716f [CI] updates to the CI report naming, and accelerate installation (#9429) 
* chore: id accordingly to avoid duplicates.

* update properly.

* updates

* updates

* empty

* updates

* changing order helps?
 2024-12-23 13:02:16 +05:30
suzukimain
753f0edff2 [docs] Replace runwayml/stable-diffusion-v1-5 with Lykon/dreamshaper-8 (#9428) 
* [docs] Replace runwayml/stable-diffusion-v1-5 with Lykon/dreamshaper-8

Updated documentation as runwayml/stable-diffusion-v1-5 has been removed from Huggingface.

* Update docs/source/en/using-diffusers/inpaint.md

Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>

* Replace with stable-diffusion-v1-5/stable-diffusion-v1-5

* Update inpaint.md

---------

Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
 2024-12-23 13:02:16 +05:30
Yuxuan.Zhang
75682c97cf CogVideoX-5b-I2V support (#9418) 
* draft Init

* draft

* vae encode image

* make style

* image latents preparation

* remove image encoder from conversion script

* fix minor bugs

* make pipeline work

* make style

* remove debug prints

* fix imports

* update example

* make fix-copies

* add fast tests

* fix import

* update vae

* update docs

* update image link

* apply suggestions from review

* apply suggestions from review

* add slow test

* make use of learned positional embeddings

* apply suggestions from review

* doc change

* Update convert_cogvideox_to_diffusers.py

* make style

* final changes

* make style

* fix tests

---------

Co-authored-by: Aryan <aryan@huggingface.co>
 2024-12-23 13:02:16 +05:30
Sayak Paul
091ad8e647 [CI] make runner_type restricted. (#9441) 
make runner_type restricted.
 2024-12-23 13:02:16 +05:30
Aryan
c2d23fbac6 Allow max shard size to be specified when saving pipeline (#9440) 
allow max shard size to be specified when saving pipeline
 2024-12-23 13:02:16 +05:30
Linoy Tsaban
b8f01f5568 [Flux Dreambooth lora] add latent caching (#9160) 
* add ostris trainer to README & add cache latents of vae

* add ostris trainer to README & add cache latents of vae

* style

* readme

* add test for latent caching

* add ostris noise scheduler
9ee1ef2a0a/toolkit/samplers/custom_flowmatch_sampler.py (L95)

* style

* fix import

* style

* fix tests

* style

* --change upcasting of transformer?

* update readme according to main

* keep only latent caching

* add configurable param for final saving of trained layers- --upcast_before_saving

* style

* Update examples/dreambooth/README_flux.md

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>

* Update examples/dreambooth/README_flux.md

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>

* use clear_objs_and_retain_memory from utilities

* style

---------

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
 2024-12-23 13:02:16 +05:30
Leo Jiang
64dd1c2f2a Fix the issue on sd3 dreambooth w./w.t. lora training (#9419) 
* Fix dtype error

* [bugfix] Fixed the issue on sd3 dreambooth training

* [bugfix] Fixed the issue on sd3 dreambooth training

---------

Co-authored-by: 蒋硕 <jiangshuo9@h-partners.com>
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
 2024-12-23 13:02:16 +05:30
Benjamin Bossan
a76767b684 MAINT Permission for GH token in stale.yml (#9427) 
* MAINT Permission for GH token in stale.yml

See https://github.com/huggingface/peft/pull/2061 for the equivalent PR
in PEFT.

This restores the functionality of the stale bot after permissions for
the token have been limited. The action still shows errors for PEFT but
the bot appears to work fine.

* Also add write permissions for PRs

---------

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
 2024-12-23 13:02:16 +05:30
Sayak Paul
457ff5dbf8 [LoRA] fix adapter movement when using DoRA. (#9411) 
fix adapter movement when using DoRA.
 2024-12-23 13:02:16 +05:30
Dhruv Nair
4bc1d700cc [CI] Nightly Test Updates (#9380) 
* update

* update

* update

* update

* update

---------

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
Co-authored-by: YiYi Xu <yixu310@gmail.com>
 2024-12-23 13:02:16 +05:30
Sayak Paul
ca1692c2c4 [CI] update artifact uploader version (#9426) 
update artifact uploader version
 2024-12-23 13:02:16 +05:30
Juan Acevedo
04abe73b30 Ptxla sd training (#9381) 
* enable pxla training of stable diffusion 2.x models.

* run linter/style and run pipeline test for stable diffusion and fix issues.

* update xla libraries

* fix read me newline.

* move files to research folder.

* update per comments.

* rename readme.

---------

Co-authored-by: Juan Acevedo <jfacevedo@google.com>
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
 2024-12-23 13:02:16 +05:30
Aryan
5e6901d361 [docs] AnimateDiff FreeNoise (#9414) 
* update docs

* apply suggestions from review

* Update docs/source/en/api/pipelines/animatediff.md

Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>

* Update docs/source/en/api/pipelines/animatediff.md

Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>

* Update docs/source/en/api/pipelines/animatediff.md

Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>

* apply suggestions from review

---------

Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
 2024-12-23 13:02:16 +05:30
asfiyab-nvidia
c35b0f7dc4 FluxPosEmbed: Remove Squeeze No-op (#9409) 
Remove Squeeze op

Signed-off-by: Asfiya Baig <asfiyab@nvidia.com>
Co-authored-by: YiYi Xu <yixu310@gmail.com>
 2024-12-23 13:02:16 +05:30
dianyo
e77c063247 Migrate the BrownianTree to BrownianInterval in DPM solver (#9335) 
migrate the BrownianTree to BrownianInterval

Co-authored-by: YiYi Xu <yixu310@gmail.com>
 2024-12-23 13:02:16 +05:30
Yu Zheng
e587cb9866 [examples] add controlnet sd3 example (#9249) 
* add controlnet sd3 example

* add controlnet sd3 example

* update controlnet sd3 example

* add controlnet sd3 example test

* fix quality and style

* update test

* update test

---------

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
 2024-12-23 13:02:16 +05:30
Sayak Paul
6f39b2bf0d [Tests] fix some fast gpu tests. (#9379) 
fix some fast gpu tests.
 2024-12-23 13:02:16 +05:30
captainzz
faa57c4ff5 fix from_transformer() with extra conditioning channels (#9364) 
* fix from_transformer() with extra conditioning channels

* style fix

---------

Co-authored-by: YiYi Xu <yixu310@gmail.com>
Co-authored-by: Álvaro Somoza <somoza.alvaro@gmail.com>
 2024-12-23 13:02:16 +05:30
Jinzhe Pan
9f011c088b [docs] Add xDiT in section optimization (#9365) 
* docs: add xDiT to optimization methods

* fix: picture layout problem

* docs: add more introduction about xdit & apply suggestions

* Apply suggestions from code review

Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>

---------

Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
 2024-12-23 13:02:16 +05:30
Igor Filippov
e8d15b22d1 [Pipeline] animatediff + vid2vid + controlnet (#9337) 
* add animatediff + vid2vide + controlnet

* post tests fixes

* PR discussion fixes

* update docs

* change input video to links on HF + update an example

* make quality fix

* fix ip adapter test

* fix ip adapter test input

* update ip adapter test
 2024-12-23 13:02:16 +05:30
YiYi Xu
04809b0ccb refactor get_timesteps for SDXL img2img + add set_begin_index (#9375) 
* refator + add begin_index

* add kolors img2img to doc
 2024-12-23 13:02:16 +05:30
Sayak Paul
b1571b6382 modify benchmarks to replace sdv1.5 with dreamshaper. (#9334) 2024-12-23 13:02:16 +05:30
YiYi Xu
d5a529773a add flux inpaint + img2img + controlnet to auto pipeline (#9367) 2024-12-23 13:02:16 +05:30
Dhruv Nair
5af81b7ce4 [CI] Quick fix for Cog Video Test (#9373) 
update
 2024-12-23 13:02:16 +05:30
Aryan
402f547d78 [core] Freenoise memory improvements (#9262) 
* update

* implement prompt interpolation

* make style

* resnet memory optimizations

* more memory optimizations; todo: refactor

* update

* update animatediff controlnet with latest changes

* refactor chunked inference changes

* remove print statements

* update

* chunk -> split

* remove changes from incorrect conflict resolution

* remove changes from incorrect conflict resolution

* add explanation of SplitInferenceModule

* update docs

* Revert "update docs"

This reverts commit c55a50a271.

* update docstring for freenoise split inference

* apply suggestions from review

* add tests

* apply suggestions from review
 2024-12-23 13:02:16 +05:30
Haruya Ishikawa
e48213cdc8 fix one uncaught deprecation warning for accessing vae_latent_channels in VaeImagePreprocessor (#9372) 
deprecation warning vae_latent_channels
 2024-12-23 13:02:16 +05:30
Linoy Tsaban
8e5940ea62 improve README for flux dreambooth lora (#9290) 
* improve readme

* improve readme

* improve readme

* improve readme
 2024-12-23 13:02:16 +05:30
Dhruv Nair
5c02d93aac [CI] Update Single file Nightly Tests (#9357) 
* update

* update
 2024-12-23 13:02:16 +05:30
Tolga Cangöz
b4ab5b0ed7 Update UNet2DConditionModel's error messages (#9230) 
* refactor
 2024-12-23 13:02:16 +05:30
Vishnu V Jaddipal
2a72888dc0 Add Flux inpainting and Flux Img2Img (#9135) 
---------

Co-authored-by: yiyixuxu <yixu310@gmail.com>
 2024-12-23 13:02:16 +05:30
Fanli Lin
704226ef68 [tests] make 2 tests device-agnostic (#9347) 
* enabel on xpu

* fix style
 2024-12-23 13:02:16 +05:30
Eduardo Escobar
58f598160f Enable load_lora_weights for StableDiffusion3InpaintPipeline (#9330) 
Enable load_lora_weights for StableDiffusion3InpaintPipeline

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
 2024-12-23 13:02:16 +05:30
YiYi Xu
ce9b8ac30b small update on rotary embedding (#9354) 
* update

* fix

---------

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
 2024-12-23 13:02:16 +05:30