LLM/diffusers
1
0
Fork 0
mirror of https://github.com/huggingface/diffusers.git synced 2026-02-15 23:37:08 +08:00
Code Issues Packages Projects Releases Wiki Activity
4,023 Commits 742 Branches 94 Tags
5fcb90f1802cf8d135cde41685da22aa49f3dbf0
Commit Graph

4023 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Jenyuan-Huang
5fcb90f180 Update InstantStyle usage in IP-Adapter documentation (#7806) 
* enable control ip-adapter per-transformer block on-the-fly


---------

Co-authored-by: sayakpaul <spsayakpaul@gmail.com>
Co-authored-by: ResearcherXman <xhs.research@gmail.com>
Co-authored-by: YiYi Xu <yixu310@gmail.com>
 2024-12-23 13:02:11 +05:30
Sayak Paul
0211759f5c [Scheduler] introduce sigma schedule. (#7649) 
* introduce sigma schedule.

Co-authored-by: Suraj Patil <surajp815@gmail.com>

* address yiyi

* update docstrings.

* implement the schedule for EDMDPMSolverMultistepScheduler

---------

Co-authored-by: Suraj Patil <surajp815@gmail.com>
 2024-12-23 13:02:11 +05:30
39th president of the United States, probably
d5f2a8c88c Add DREAM training (#6381) 
A new function compute_dream_and_update_latents has been added to the
training utilities that allows you to do DREAM rectified training in line
with the paper https://arxiv.org/abs/2312.00210. The method can be used
with an extra argument in the train_text_to_image.py script.

Co-authored-by: Jimmy <39@🇺🇸.com>
 2024-12-23 13:02:11 +05:30
Fabio Rigano
ec8ce0c2a0 [Docs] Update image masking and face id example (#7780) 
* [Docs] Update image masking and face id example

* Update docs

* Fix docs
 2024-12-23 13:02:11 +05:30
Beinsezii
b43b87d507 Add PixArtSigmaPipeline to AutoPipeline mapping (#7783) 2024-12-23 13:02:11 +05:30
Steven Liu
bf1957f6d4 [docs] Fix AutoPipeline docstring (#7779) 
fix

Co-authored-by: YiYi Xu <yixu310@gmail.com>
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
 2024-12-23 13:02:11 +05:30
btlorch
4188f306c4 Convert RGB to BGR for the SDXL watermark encoder (#7013) 
* Convert channel order to BGR for the watermark encoder. Convert the watermarked BGR images back to RGB. Fixes #6292

* Revert channel order before stacking images to overcome limitations that negative strides are currently not supported

---------

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
 2024-12-23 13:02:11 +05:30
Steven Liu
de414618ba [docs] Refactor image quality docs (#7758) 
* refactor

* code snippets

* fix path

* fix path in guide

* code outputs

* align toctree title

* title

* fix title
 2024-12-23 13:02:11 +05:30
Steven Liu
bebfb61c5c [docs] Reproducible pipelines (#7769) 
* reproducibility

* feedback

* feedback

* fix path

* github link
 2024-12-23 13:02:11 +05:30
Sayak Paul
513fea53eb Fix lora device test (#7738) 
* fix lora device test

* fix more.

* fix more/

* quality

* empty

---------

Co-authored-by: Dhruv Nair <dhruv.nair@gmail.com>
 2024-12-23 13:02:11 +05:30
Sayak Paul
8373a61a95 [Tests] mark UNetControlNetXSModelTests::test_forward_no_control to be flaky (#7771) 
decorate UNetControlNetXSModelTests::test_forward_no_control with is_flaky
 2024-12-23 13:02:11 +05:30
Sayak Paul
b4d6d464d0 [PixArt] fix small nits in pixart sigma (#7767) 
fix small nits in pixart sigma
 2024-12-23 13:02:11 +05:30
Junsong Chen
55d7ad64ea PixArt-Sigma Implementation (#7654) 
* support PixArt-DMD

---------

Co-authored-by: jschen <chenjunsong4@h-partners.com>
Co-authored-by: badayvedat <badayvedat@gmail.com>
Co-authored-by: Vedat Baday <54285744+badayvedat@users.noreply.github.com>
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
Co-authored-by: YiYi Xu <yixu310@gmail.com>
Co-authored-by: yiyixuxu <yixu310@gmail,com>
 2024-12-23 13:02:11 +05:30
Dhruv Nair
9de95b0b96 Fix test for consistency decoder. (#7746) 
update
 2024-12-23 13:02:11 +05:30
Dhruv Nair
77d3449200 Fix failing VAE tiling test (#7747) 
update
 2024-12-23 13:02:11 +05:30
Steven Liu
ed20a5ac49 [docs] Clean up toctree (#7715) 
* toctree

* optim

* feedback

* improve overview
 2024-12-23 13:02:11 +05:30
Sayak Paul
9bef15a567 [Metadat utils] fix: json lines ordering. (#7744) 
fix: json lines ordering.
 2024-12-23 13:02:11 +05:30
Sai-Suraj-27
5e66907fda fix: Fixed a wrong decorator by modifying it to @classmethod (#7653) 
* Fixed wrong decorator by modifying it to @classmethod.

* Updated the method and it's argument.

---------

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
 2024-12-23 13:02:10 +05:30
Fabio Rigano
030b9de0df Restore AttnProcessor2_0 in unload_ip_adapter (#7727) 
* Restore AttnProcessor2_0 in unload_ip_adapter

* Fix style

* Update test

---------

Co-authored-by: YiYi Xu <yixu310@gmail.com>
 2024-12-23 13:02:10 +05:30
Jenyuan-Huang
4ec19bc3d5 Support InstantStyle (#7668) 
* enable control ip-adapter per-transformer block on-the-fly

---------

Co-authored-by: sayakpaul <spsayakpaul@gmail.com>
Co-authored-by: ResearcherXman <xhs.research@gmail.com>
Co-authored-by: YiYi Xu <yixu310@gmail.com>
 2024-12-23 13:02:10 +05:30
Phil Butler
26f6b4c596 Remove redundant lines (#7396) 
Co-authored-by: Dhruv Nair <dhruv.nair@gmail.com>
Co-authored-by: YiYi Xu <yixu310@gmail.com>
 2024-12-23 13:02:10 +05:30
Steven Liu
1cce2c1c25 [docs] AutoPipeline (#7714) 
* autopipeline

* edits

* feedback

---------

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
 2024-12-23 13:02:10 +05:30
Dhruv Nair
4b518b4a6e Fix Kandinksy V22 tests (#7699) 
update
 2024-12-23 13:02:10 +05:30
Dhruv Nair
6e888b0a69 Update Wuerschten Test (#7700) 
update
 2024-12-23 13:02:10 +05:30
Abhinav Gopal
024fb640e0 Animatediff Controlnet Community Pipeline IP Adapter Fix (#7413) 
* fixed encode_image function signature in controlnet animatediff

* copied encode_image from stable diffusion pipeline

---------

Co-authored-by: YiYi Xu <yixu310@gmail.com>
 2024-12-23 13:02:10 +05:30
Guillaume LEGENDRE
3cf49fd38f fix/add tailscale key in case of failure (#7719) 
add tailscale key in case of failure
 2024-12-23 13:02:10 +05:30
Sai-Suraj-27
20b5919ab2 fix: Fixed type annotations for compatability with python 3.8 (#7648) 
* Fixed type annotations for compatability with python 3.8

* Add required imports.
 2024-12-23 13:02:10 +05:30
Dhruv Nair
eb75ed9110 Cleanup ControlnetXS (#7701) 
* update

* update
 2024-12-23 13:02:10 +05:30
Dhruv Nair
5f5c806d9c Cast height, width to int inside prepare latents (#7691) 
update
 2024-12-23 13:02:10 +05:30
YiYi Xu
cd4743d978 adding back test_conversion_when_using_device_map (#7704) 
* style


* Fix device map nits (#7705)


---------

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
 2024-12-23 13:02:10 +05:30
Fabio Rigano
c5a2b97bff Move IP Adapter Face ID to core (#7186) 
* Switch to peft and multi proj layers

* Move Face ID loading and inference to core

---------

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
 2024-12-23 13:02:10 +05:30
Guillaume LEGENDRE
bb07c28442 Add tailscale action to push_test (#7709) 2024-12-23 13:02:10 +05:30
Steven Liu
2bed2f4c45 [docs] Pipeline loading (#7684) 
* pipelines

* schedulers and models

* community pipelines

* feedback
 2024-12-23 13:02:10 +05:30
Wentian
ff3e9587d9 [Docs] Update TGATE in section optimization. (#7698) 
Update tgate.md
 2024-12-23 13:02:10 +05:30
Sayak Paul
d01453e437 [Workflows] remove installation of redundant modules from flax PR tests (#7662) 
remove installation of redundant modules from flax PR tests
 2024-12-23 13:02:10 +05:30
Dhruv Nair
d166f496b6 Don't install PEFT with UV in slow tests (#7697) 
* update

* update
 2024-12-23 13:02:10 +05:30
Sai-Suraj-27
2b16db59f1 fix: Updated ruff configuration to avoid deprecated configuration warning (#7637) 
Updated ruff configuration to avoid depreceated config.

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
 2024-12-23 13:02:10 +05:30
Wentian
92125c6887 [Docs] Add TGATE in section optimization (#7639) 
* Create tgate.md

* Update _toctree.yml

* Update docs/source/en/optimization/tgate.md

Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>

* Update docs/source/en/optimization/tgate.md

Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>

* Update docs/source/en/optimization/tgate.md

Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>

* Update docs/source/en/optimization/tgate.md

Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>

* Update docs/source/en/optimization/tgate.md

Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>

* Update docs/source/en/optimization/tgate.md

Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>

* Update docs/source/en/optimization/tgate.md

Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>

* Update docs/source/en/optimization/tgate.md

Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>

* Update docs/source/en/optimization/tgate.md

Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>

* Update docs/source/en/optimization/tgate.md

Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>

* Update tgate.md

* Update tgate.md

---------

Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
 2024-12-23 13:02:10 +05:30
Sayak Paul
4e28bea1bc [Core] is_cosxl_edit arg in SDXL ip2p. (#7650) 
* is_cosxl_edit arg in SDXL ip2p.

* Empty-Commit

Co-authored-by: Yiyi Xu <yixu310@gmail.com>

* doc

* remove redundant logic.

* reflect drhuv's comments.

---------

Co-authored-by: Yiyi Xu <yixu310@gmail.com>
Co-authored-by: Dhruv Nair <dhruv.nair@gmail.com>
 2024-12-23 13:02:10 +05:30
UmerHA
d7895dfa9d Fixing implementation of ControlNet-XS (#6772) 
* CheckIn - created DownSubBlocks

* Added extra channels, implemented subblock fwd

* Fixed connection sizes

* checkin

* Removed iter, next in forward

* Models for SD21 & SDXL run through

* Added back pipelines, cleared up connections

* Cleaned up connection creation

* added debug logs

* updated logs

* logs: added input loading

* Update umer_debug_logger.py

* log: Loading hint

* Update umer_debug_logger.py

* added logs

* Changed debug logging

* debug: added more logs

* Fixed num_norm_groups

* Debug: Logging all of SDXL input

* Update umer_debug_logger.py

* debug: updated logs

* checkim

* Readded tests

* Removed debug logs

* Fixed Slow Tests

* Added value ckecks | Updated model_cpu_offload_seq

* accelerate-offloading works ; fast tests work

* Made unet & addon explicit in controlnet

* Updated slow tests

* Added dtype/device to ControlNetXS

* Filled in test model paths

* Added image_encoder/feature_extractor to XL pipe

* Fixed fast tests

* Added comments and docstrings

* Fixed copies

* Added docs ; Updates slow tests

* Moved changes to UNetMidBlock2DCrossAttn

* tiny cleanups

* Removed stray prints

* Removed ip adapters + freeU

- Removed ip adapters + freeU as they don't make sense for ControlNet-XS
- Fixed imports of UNet components

* Fixed test_save_load_float16

* Make style, quality, fix-copies

* Changed loading/saving API for ControlNetXS

- Changed loading/saving API for ControlNetXS
- other small fixes

* Removed ControlNet-XS from research examples

* Make style, quality, fix-copies

* Small fixes

- deleted ControlNetXSModel.init_original
- added time_embedding_mix to StableDiffusionControlNetXSPipeline .from_pretrained / StableDiffusionXLControlNetXSPipeline.from_pretrained
- fixed copy hints

* checkin May 11 '23

* CheckIn Mar 12 '24

* Fixed tests for SD

* Added tests for UNetControlNetXSModel

* Fixed SDXL tests

* cleanup

* Delete Pipfile

* CheckIn Mar 20

Started replacing sub blocks  by `ControlNetXSCrossAttnDownBlock2D` and `ControlNetXSCrossAttnUplock2D`

* check-in Mar 23

* checkin 24 Mar

* Created init for UNetCnxs and CnxsAddon

* CheckIn

* Made from_modules, from_unet and no_control work

* make style,quality,fix-copies & small changes

* Fixed freezing

* Added gradient ckpt'ing; fixed tests

* Fix slow tests(+compile) ; clear naming confusion

* Don't create UNet in init ; removed class_emb

* Incorporated review feedback

- Deleted get_base_pipeline /  get_controlnet_addon for pipes
- Pipes inherit from StableDiffusionXLPipeline
- Made module dicts for cnxs-addon's down/mid/up classes
- Added support for qkv fusion and freeU

* Make style, quality, fix-copies

* Implemented review feedback

* Removed compatibility check for vae/ctrl embedding

* make style, quality, fix-copies

* Delete Pipfile

* Integrated review feedback

- Importing ControlNetConditioningEmbedding now
- get_down/mid/up_block_addon now outside class
- renamed `do_control` to `apply_control`

* Reduced size of test tensors

For this, added `norm_num_groups` as parameter everywhere

* Renamed cnxs-`Addon` to cnxs-`Adapter`

- `ControlNetXSAddon` -> `ControlNetXSAdapter`
- `ControlNetXSAddonDownBlockComponents` -> `DownBlockControlNetXSAdapter`, and similarly for mid/up
- `get_mid_block_addon` -> `get_mid_block_adapter`, and similarly for mid/up

* Fixed save_pretrained/from_pretrained bug

* Removed redundant code

---------

Co-authored-by: Dhruv Nair <dhruv.nair@gmail.com>
 2024-12-23 13:02:10 +05:30
Sayak Paul
5301343fc9 don't install peft from the source with uv for now. (#7679) 2024-12-23 13:02:10 +05:30
Sayak Paul
a761390f12 fix: metadata token (#7631) 2024-12-23 13:02:10 +05:30
Sayak Paul
ab34f394df make docker-buildx mandatory. (#7652) 2024-12-23 13:02:10 +05:30
kabachuha
679d7231e7 Add (Scheduled) Pseudo-Huber Loss training scripts to research projects (#7527) 
* add scheduled pseudo-huber loss training scripts

See #7488

* add reduction modes to huber loss

* [DB Lora] *2 multiplier to huber loss cause of 1/2 a^2 conv.

pairing of c6495def1f

* [DB Lora] add option for smooth l1 (huber / delta)

Pairing of dd22958caa

* [DB Lora] unify huber scheduling

Pairing of 19a834c3ab

* [DB Lora] add snr huber scheduler

Pairing of 47fb1a6854

* fixup examples link

* use snr schedule by default in DB

* update all huber scripts with snr

* code quality

* huber: make style && make quality

---------

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
 2024-12-23 13:02:10 +05:30
Benjamin Bossan
a0662fdb27 FIX Setting device for DoRA parameters (#7655) 
Fix a bug that causes the the call to set_lora_device to ignore the DoRA
parameters.
 2024-12-23 13:02:10 +05:30
Sai-Suraj-27
499d96bfd1 fix: Replaced deprecated logger.warn with logger.warning (#7643) 
Fixed deprecated logger.warn with logger.warning.
 2024-12-23 13:02:10 +05:30
Yiqin Zhao
d47de39a46 Fixed YAML loading. (#7579) 2024-12-23 13:02:10 +05:30
dg845
6edad9389c LCM Distill Scripts Fix Bug when Initializing Target U-Net (#6848) 
* Initialize target_unet from unet rather than teacher_unet so that we correctly add time_embedding.cond_proj if necessary.

* Use UNet2DConditionModel.from_config to initialize target_unet from unet's config.

---------

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
 2024-12-23 13:02:10 +05:30
Steven Munn
9fffbee44f Skip PEFT LoRA Scaling if the scale is 1.0 (#7576) 
* Skip scaling if scale is identity

* move check for weight one to scale and unscale lora

* fix code style/quality

* Empty-Commit

---------

Co-authored-by: Steven Munn <stevenjmunn@gmail.com>
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
Co-authored-by: Steven Munn <5297082+stevenjlm@users.noreply.github.com>
 2024-12-23 13:02:10 +05:30
Sayak Paul
42dab12780 [Core] fix img2img pipeline for Playground (#7627) 
* playground vae encoding should use std and mean of the vae.

* style.

* fix-copies.
 2024-12-23 13:02:10 +05:30