LLM/diffusers
1
0
Fork 0
mirror of https://github.com/huggingface/diffusers.git synced 2026-02-17 00:06:20 +08:00
Code Issues Packages Projects Releases Wiki Activity
4,913 Commits 745 Branches 94 Tags
4c5082ca685d1dd0a24907890da170e258e5cd01
Commit Graph

4913 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Aryan
f6b690301d [Pipeline] AnimateDiff SDXL (#6721) 
* update conversion script to handle motion adapter sdxl checkpoint

* add animatediff xl

* handle addition_embed_type

* fix output

* update

* add imports

* make fix-copies

* add decode latents

* update docstrings

* add animatediff sdxl to docs

* remove unnecessary lines

* update example

* add test

* revert conv_in conv_out kernel param

* remove unused param addition_embed_type_num_heads

* latest IPAdapter impl

* make fix-copies

* fix return

* add IPAdapterTesterMixin to tests

* fix return

* revert based on suggestion

* add freeinit

* fix test_to_dtype test

* use StableDiffusionMixin instead of different helper methods

* fix progress bar iterations

* apply suggestions from review

* hardcode flip_sin_to_cos and freq_shift

* make fix-copies

* fix ip adapter implementation

* fix last failing test

* make style

* Update docs/source/en/api/pipelines/animatediff.md

Co-authored-by: Dhruv Nair <dhruv.nair@gmail.com>

* remove todo

* fix doc-builder errors

---------

Co-authored-by: Dhruv Nair <dhruv.nair@gmail.com>
 2024-12-23 13:02:11 +05:30
Philip Pham
23a5119e93 Check shape and remove deprecated APIs in scheduling_ddpm_flax.py (#7703) 
`model_output.shape` may only have rank 1.

There are warnings related to use of random keys.

```
tests/schedulers/test_scheduler_flax.py: 13 warnings
  /Users/phillypham/diffusers/src/diffusers/schedulers/scheduling_ddpm_flax.py:268: FutureWarning: normal accepts a single key, but was given a key array of shape (1, 2) != (). Use jax.vmap for batching. In a future JAX version, this will be an error.
    noise = jax.random.normal(split_key, shape=model_output.shape, dtype=self.dtype)

tests/schedulers/test_scheduler_flax.py::FlaxDDPMSchedulerTest::test_betas
  /Users/phillypham/virtualenv/diffusers/lib/python3.9/site-packages/jax/_src/random.py:731: FutureWarning: uniform accepts a single key, but was given a key array of shape (1,) != (). Use jax.vmap for batching. In a future JAX version, this will be an error.
    u = uniform(key, shape, dtype, lo, hi)  # type: ignore[arg-type]
```
 2024-12-23 13:02:11 +05:30
Tolga Cangöz
c8a489399c Fix image upcasting (#7858) 
Fix image's upcasting before `vae.encode()` when using `fp16`

Co-authored-by: YiYi Xu <yixu310@gmail.com>
 2024-12-23 13:02:11 +05:30
Hyoungwon Cho
ee1b41e1fe Modification on the PAG community pipeline (re) (#7876) 
* edited_pag_implementation

* update

---------

Co-authored-by: yiyixuxu <yixu310@gmail.com>
 2024-12-23 13:02:11 +05:30
Bagheera
18aedaa230 7879 - adjust documentation to use naruto dataset, since pokemon is now gated (#7880) 
* 7879 - adjust documentation to use naruto dataset, since pokemon is now gated

* replace references to pokemon in docs

* more references to pokemon replaced

* Japanese translation update

---------

Co-authored-by: bghira <bghira@users.github.com>
 2024-12-23 13:02:11 +05:30
Álvaro Somoza
a6b6658dee Fix for "no lora weight found module" with some loras (#7875) 
* return layer weight if not found

* better system and test

* key example and typo
 2024-12-23 13:02:11 +05:30
Steven Liu
f9c78fc6f8 [docs] Distilled inference (#7834) 
* combine

* edits
 2024-12-23 13:02:11 +05:30
Guillaume LEGENDRE
8d4f8cb2de Ci - change cache folder (#7867) 2024-12-23 13:02:11 +05:30
Steven Liu
e2d7831b8f [docs] LCM (#7829) 
* lcm

* lcm lora

* fix

* fix hfoption

* edits
 2024-12-23 13:02:11 +05:30
HelloWorldBeginner
93d9d2f9bd Add Ascend NPU support for SDXL fine-tuning and fix the model saving bug when using DeepSpeed. (#7816) 
* Add Ascend NPU support for SDXL fine-tuning and fix the model saving bug when using DeepSpeed.

* fix check code quality

* Decouple the NPU flash attention and make it an independent module.

* add doc and unit tests for npu flash attention.

---------

Co-authored-by: mhh001 <mahonghao1@huawei.com>
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
 2024-12-23 13:02:11 +05:30
Dhruv Nair
18a5b2d98f Remove installing python again in container (#7852) 
update
 2024-12-23 13:02:11 +05:30
Lucain
4a008a7b30 Respect resume_download deprecation (#7843) 
* Deprecate resume_download

* align docstring with transformers

* style

---------

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
 2024-12-23 13:02:11 +05:30
Aritra Roy Gosthipaty
4f985aa84d [Tests] reduce the model size in the blipdiffusion fast test (#7849) 
reducing model size
 2024-12-23 13:02:11 +05:30
Dhruv Nair
ec38dacf45 Update deps for pipe test fetcher (#7838) 
update

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
 2024-12-23 13:02:11 +05:30
Guillaume LEGENDRE
585f1f7982 Change GPU Runners (#7840) 
* Move to new GPU Runners for slow tests

* Move to new GPU Runners for nightly tests
 2024-12-23 13:02:11 +05:30
Guillaume LEGENDRE
5daae497e2 move to new runners (#7839) 2024-12-23 13:02:11 +05:30
Dhruv Nair
683cf0cc1f Update CI cache (#7832) 
update

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
 2024-12-23 13:02:11 +05:30
Dhruv Nair
341c3e590e Update download diff format tests (#7831) 
update

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
 2024-12-23 13:02:11 +05:30
Dhruv Nair
affaa1832a Fix hanging pipeline fetching (#7837) 
update
 2024-12-23 13:02:11 +05:30
yunseong Cho
f47aecb8a4 Fix key error for dictionary with randomized order in convert_ldm_unet_checkpoint (#7680) 
fix key error for different order

Co-authored-by: yunseong <yunseong.cho@superlabs.us>
Co-authored-by: Dhruv Nair <dhruv.nair@gmail.com>
 2024-12-23 13:02:11 +05:30
Aritra Roy Gosthipaty
de4272e93e [Tests] reduce the model size in the audioldm fast test (#7833) 
chore: initial size reduction of models
 2024-12-23 13:02:11 +05:30
YiYi Xu
2694cf960b [ip-adapter] fix ip-adapter for StableDiffusionInstructPix2PixPipeline (#7820) 
update prepare_ip_adapter_ for pix2pix
 2024-12-23 13:02:11 +05:30
YiYi Xu
e560f7a6b8 update the logic of is_sequential_cpu_offload (#7788) 
* up

* add comment to the tests + fix dit

---------

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
 2024-12-23 13:02:11 +05:30
Sayak Paul
87b40ac14c [Tests] fix: device map tests for models (#7825) 
* fix: device module tests

* remove patch file

* Empty-Commit
 2024-12-23 13:02:11 +05:30
Dhruv Nair
c0410e4d14 Fix for pipeline slow test fetcher (#7824) 
* update

* update
 2024-12-23 13:02:11 +05:30
Steven Liu
18f67e82d8 [docs] Community pipelines (#7819) 
* community pipelines

* feedback

* consolidate
 2024-12-23 13:02:11 +05:30
Tolga Cangöz
99b7ee9fd8 Fix CPU offload in docstring (#7827) 
Fix cpu offload
 2024-12-23 13:02:11 +05:30
Dhruv Nair
eccd1d3701 SSH Runner Workflow Update (#7822) 
* add debug workflow

* update
 2024-12-23 13:02:11 +05:30
Linoy Tsaban
0a81020e53 Add B-Lora training option to the advanced dreambooth lora script (#7741) 
* add blora

* add blora

* add blora

* add blora

* little changes

* little changes

* remove redundancies

* fixes

* add B LoRA to readme

* style

* inference

* defaults + path to loras+ generation

* minor changes

* style

* minor changes

* minor changes

* blora arg

* added --lora_unet_blocks

* style

* Update examples/advanced_diffusion_training/README.md

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>

* add commit hash to B-LoRA repo cloneing

* change inference, remove cloning

* change inference, remove cloning
add section about configureable unet blocks

* change inference, remove cloning
add section about configureable unet blocks

* Apply suggestions from code review

---------

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
 2024-12-23 13:02:11 +05:30
Sayak Paul
970d780dbb [Core] introduce _no_split_modules to ModelMixin (#6396) 
* introduce _no_split_modules.

* unnecessary spaces.

* remove unnecessary kwargs and style

* fix: accelerate imports.

* change to _determine_device_map

* add the blocks that have residual connections.

* add: CrossAttnUpBlock2D

* add: testin

* style

* line-spaces

* quality

* add disk offload test without safetensors.

* checking disk offloading percentages.

* change model split

* add: utility for checking multi-gpu requirement.

* model parallelism test

* splits.

* splits.

* splits

* splits.

* splits.

* splits.

* offload folder to test_disk_offload_with_safetensors

* add _no_split_modules

* fix-copies
 2024-12-23 13:02:11 +05:30
Aritra Roy Gosthipaty
6561e9d5e7 [Tests] reduce the model size in the amused fast test (#7804) 
* chore: reducing model sizes

* chore: shrinks further

* chore: shrinks further

* chore: shrinking model for img2img pipeline

* chore: reducing size of model for inpaint pipeline

---------

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
 2024-12-23 13:02:11 +05:30
Aritra Roy Gosthipaty
2ba3356cbe [Tests] reduce the model size in the ddpm fast test (#7797) 
* chore: reducing unet size for faster tests

* review suggestions

---------

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
 2024-12-23 13:02:11 +05:30
Aritra Roy Gosthipaty
44c27540f5 [Tests] reduce the model size in the ddim fast test (#7803) 
chore: reducing model size for ddim fast pipeline

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
 2024-12-23 13:02:11 +05:30
Clint Adams
b3b3d961a7 Set main_input_name in StableDiffusionSafetyChecker to "clip_input" (#7500) 
FlaxStableDiffusionSafetyChecker sets main_input_name to "clip_input".
This makes StableDiffusionSafetyChecker consistent.

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
Co-authored-by: YiYi Xu <yixu310@gmail.com>
 2024-12-23 13:02:11 +05:30
RuiningLi
a59468b741 Added get_velocity function to EulerDiscreteScheduler. (#7733) 
* Added get_velocity function to EulerDiscreteScheduler.

* Fix white space on blank lines

* Added copied from statement

* back to the original.

---------

Co-authored-by: Ruining Li <ruining@robots.ox.ac.uk>
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
 2024-12-23 13:02:11 +05:30
jschoormans
277a06c705 Adding TextualInversionLoaderMixin for the controlnet_inpaint_sd_xl pipeline (#7288) 
* added TextualInversionMixIn to controlnet_inpaint_sd_xl pipeline


---------

Co-authored-by: YiYi Xu <yixu310@gmail.com>
 2024-12-23 13:02:11 +05:30
Dhruv Nair
eaf01bc16a Safetensor loading in AnimateDiff conversion scripts (#7764) 
* update

* update
 2024-12-23 13:02:11 +05:30
Yushu
32a1059458 [Pipeline] Fix error of SVD pipeline when num_videos_per_prompt > 1 (#7786) 
swap the order for do_classifier_free_guidance concat with repeat

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
Co-authored-by: Dhruv Nair <dhruv.nair@gmail.com>
 2024-12-23 13:02:11 +05:30
Dhruv Nair
bd61b8c1f1 Add debugging workflow (#7778) 
add debug workflow

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
 2024-12-23 13:02:11 +05:30
Nilesh
9388258704 Check for latents, before calling prepare_latents - sdxlImg2Img (#7582) 
* Check for latents, before calling prepare_latents - sdxlImg2Img

* Added latents check for all the img2img pipeline

* Fixed silly mistake while checking latents as None
 2024-12-23 13:02:11 +05:30
Jenyuan-Huang
5fcb90f180 Update InstantStyle usage in IP-Adapter documentation (#7806) 
* enable control ip-adapter per-transformer block on-the-fly


---------

Co-authored-by: sayakpaul <spsayakpaul@gmail.com>
Co-authored-by: ResearcherXman <xhs.research@gmail.com>
Co-authored-by: YiYi Xu <yixu310@gmail.com>
 2024-12-23 13:02:11 +05:30
Sayak Paul
0211759f5c [Scheduler] introduce sigma schedule. (#7649) 
* introduce sigma schedule.

Co-authored-by: Suraj Patil <surajp815@gmail.com>

* address yiyi

* update docstrings.

* implement the schedule for EDMDPMSolverMultistepScheduler

---------

Co-authored-by: Suraj Patil <surajp815@gmail.com>
 2024-12-23 13:02:11 +05:30
39th president of the United States, probably
d5f2a8c88c Add DREAM training (#6381) 
A new function compute_dream_and_update_latents has been added to the
training utilities that allows you to do DREAM rectified training in line
with the paper https://arxiv.org/abs/2312.00210. The method can be used
with an extra argument in the train_text_to_image.py script.

Co-authored-by: Jimmy <39@🇺🇸.com>
 2024-12-23 13:02:11 +05:30
Fabio Rigano
ec8ce0c2a0 [Docs] Update image masking and face id example (#7780) 
* [Docs] Update image masking and face id example

* Update docs

* Fix docs
 2024-12-23 13:02:11 +05:30
Beinsezii
b43b87d507 Add PixArtSigmaPipeline to AutoPipeline mapping (#7783) 2024-12-23 13:02:11 +05:30
Steven Liu
bf1957f6d4 [docs] Fix AutoPipeline docstring (#7779) 
fix

Co-authored-by: YiYi Xu <yixu310@gmail.com>
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
 2024-12-23 13:02:11 +05:30
btlorch
4188f306c4 Convert RGB to BGR for the SDXL watermark encoder (#7013) 
* Convert channel order to BGR for the watermark encoder. Convert the watermarked BGR images back to RGB. Fixes #6292

* Revert channel order before stacking images to overcome limitations that negative strides are currently not supported

---------

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
 2024-12-23 13:02:11 +05:30
Steven Liu
de414618ba [docs] Refactor image quality docs (#7758) 
* refactor

* code snippets

* fix path

* fix path in guide

* code outputs

* align toctree title

* title

* fix title
 2024-12-23 13:02:11 +05:30
Steven Liu
bebfb61c5c [docs] Reproducible pipelines (#7769) 
* reproducibility

* feedback

* feedback

* fix path

* github link
 2024-12-23 13:02:11 +05:30
Sayak Paul
513fea53eb Fix lora device test (#7738) 
* fix lora device test

* fix more.

* fix more/

* quality

* empty

---------

Co-authored-by: Dhruv Nair <dhruv.nair@gmail.com>
 2024-12-23 13:02:11 +05:30