UmerHA
16e445a49f
Implements Blockwise lora ( #7352 )
...
* Initial commit
* Implemented block lora
- implemented block lora
- updated docs
- added tests
* Finishing up
* Reverted unrelated changes made by make style
* Fixed typo
* Fixed bug + Made text_encoder_2 scalable
* Integrated some review feedback
* Incorporated review feedback
* Fix tests
* Made every module configurable
* Adapter to new lora test structure
* Final cleanup
* Some more final fixes
- Included examples in `using_peft_for_inference.md`
- Added hint that only attns are scaled
- Removed NoneTypes
- Added test to check mismatching lens of adapter names / weights raise error
* Update using_peft_for_inference.md
* Update using_peft_for_inference.md
* Make style, quality, fix-copies
* Updated tutorial;Warning if scale/adapter mismatch
* floats are forwarded as-is; changed tutorial scale
* make style, quality, fix-copies
* Fixed typo in tutorial
* Moved some warnings into `lora_loader_utils.py`
* Moved scale/lora mismatch warnings back
* Integrated final review suggestions
* Empty commit to trigger CI
* Reverted emoty commit to trigger CI
---------
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
2024-12-23 13:02:10 +05:30
Sayak Paul
a48c41f4c7
add: space for calculating memory usagee. ( #7414 )
...
* add: space for calculating memory usahe.
* Update docs/source/en/using-diffusers/loading.md
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
---------
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
2024-12-23 13:02:09 +05:30
sayakpaul
fec596ddf1
Revert "add: space within docs to calculate mememory usage."
...
This reverts commit 78990dd960
2024-12-23 13:02:09 +05:30
sayakpaul
634c467193
add: space within docs to calculate mememory usage.
2024-12-23 13:02:09 +05:30
M. Tolga Cangöz
651dac5447
Fix typos ( #7411 )
...
* Fix typos
* Fix typo in SVD.md
2024-12-23 13:02:09 +05:30
Sayak Paul
8d9dadaa64
[Custom Pipelines with Custom Components] fix multiple things ( #7304 )
...
* checking to improve pipelines.
* more fixes.
* add: tip to encourage the usage of revision
* Apply suggestions from code review
* retrigger ci
---------
Co-authored-by: YiYi Xu <yixu310@gmail.com >
2024-12-23 13:02:09 +05:30
Michael
f08278b391
Add Intro page of TCD ( #7259 )
...
* add tcd intro
* resolve repos
* Apply suggestions from code review
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
* revise NFEs related
* change inpainting location
---------
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
2024-12-23 13:02:09 +05:30
UmerHA
c5c113369b
Adds denoising_end parameter to ControlNetPipeline for SDXL ( #6175 )
...
* Initial commit
* Removed copy hints, as in original SDXLControlNetPipeline
Removed copy hints, as in original SDXLControlNetPipeline, as the `make fix-copies` seems to have issues with the @property decorator.
* Reverted changes to ControlNetXS
* Addendum to: Removed changes to ControlNetXS
* Added test+docs for mixture of denoiser
* Update docs/source/en/using-diffusers/controlnet.md
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
* Update docs/source/en/using-diffusers/controlnet.md
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
---------
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
2024-12-23 13:02:09 +05:30
Steven Liu
6b2f8109bc
[docs] IP-Adapter image embedding ( #7226 )
...
* update
* fix parameter name
* feedback
* add no mask version
---------
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
2024-12-23 13:02:09 +05:30
Steven Liu
9d10e629af
[docs] Community tips ( #7137 )
...
* tips
* feedback
* callback only
2024-12-23 13:02:08 +05:30
Steven Liu
19d9e7d5d9
[docs] Merge LoRAs ( #7213 )
...
* merge loras
* feedback
* torch.compile
* feedback
2024-12-23 13:02:08 +05:30
bimsarapathiraja
e5b4915090
Remove the line. Using it create wrong output ( #7075 )
...
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
2024-12-23 13:02:08 +05:30
Vinh H. Pham
eb3c60cbde
[Docs] Update callback.md code example ( #7150 )
...
Update callback.md
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
2024-12-23 13:02:08 +05:30
M. Tolga Cangöz
06de6e0b0f
Fix typos ( #7181 )
...
* Fix typos
* Fix typos
* Fix typos and update documentation in lora.md
2024-12-23 13:02:08 +05:30
YiYi Xu
f38417fe30
[ip-adapter] refactor prepare_ip_adapter_image_embeds and skip load image_encoder ( #7016 )
...
* add
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
---------
Co-authored-by: yiyixuxu <yixu310@gmail,com>
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
2024-12-23 13:02:08 +05:30
M. Tolga Cangöz
a3c115949d
[Docs] Fix typos ( #7131 )
...
* Add copyright notice to relevant files and fix typos
* Set `timestep_spacing` parameter of `StableDiffusionXLPipeline`'s scheduler to `'trailing'`.
* Update `StableDiffusionXLPipeline.from_single_file` by including EulerAncestralDiscreteScheduler with `timestep_spacing="trailing"` param.
* Update model loading method in SDXL Turbo documentation
2024-12-23 13:02:08 +05:30
M. Tolga Cangöz
59ed616c16
[Docs] Fix typos ( #7118 )
...
Fix typos, formatting and remove trailing whitespace
2024-12-23 13:02:08 +05:30
Steven Liu
d6e432c38a
[docs] Minor updates ( #7063 )
...
* updates
* feedback
2024-12-23 13:02:08 +05:30
bimsarapathiraja
85e57454a0
Change images to image. The variable images is not used anywhere ( #7074 )
2024-12-23 13:02:08 +05:30
YiYi Xu
cae3e45cbb
fix doc example for fom_single_file ( #7015 )
...
* fix doc
* remove use_safetensors from signature
* more
---------
Co-authored-by: yiyixuxu <yixu310@gmail,com>
2024-12-23 13:02:08 +05:30
Fabio Rigano
98326b5f04
IP-Adapter attention masking ( #6847 )
...
* Add attention masking to attn processors
* Update tensor conversion
---------
Co-authored-by: YiYi Xu <yixu310@gmail.com >
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
2024-12-23 13:02:08 +05:30
Steven Liu
374cbcbd42
[docs] Video generation ( #6701 )
...
* first draft
* fix path
* fix path
* i2vgen-xl
* review
* modelscopet2v
* feedback
2024-12-23 13:02:07 +05:30
Steven Liu
0535a85d52
[docs] Fix callout ( #6998 )
...
Update ip_adapter.md
2024-12-23 13:02:07 +05:30
Sayak Paul
e42e3ef1ab
[IP Adapters] feat: allow low_cpu_mem_usage in ip adapter loading ( #6946 )
...
* feat: allow low_cpu_mem_usage in ip adapter loading
* reduce the number of device placements.
* documentation.
* throw low_cpu_mem_usage warning only once from the main entry point.
2024-12-23 13:02:07 +05:30
Steven Liu
ceb53a65bd
[docs] IP-Adapter ( #6897 )
...
* use cases
* first draft
* fix image links
* lcm-lora
* feedback
* review
* feedback
* feedback
2024-12-23 13:02:07 +05:30
Steven Liu
1b6feac509
[docs] Community pipelines ( #6929 )
...
fix
2024-12-23 13:02:07 +05:30
Sayak Paul
86e62ae4c5
change to 2024 in the license ( #6902 )
...
change to 2024
2024-12-23 13:02:07 +05:30
Félix Sanz
6d0fe4ea94
fix: typo in callback function name and property ( #6834 )
...
* fix: callback function name is incorrect
On this tutorial there is a function defined and then used inside `callback_on_step_end` argument, but the name was not correct (mismatch)
* fix: typo in num_timestep (correct is num_timesteps)
fixed property name
2024-12-23 13:02:06 +05:30
YiYi Xu
f1e105e664
[IP-Adapter] Support multiple IP-Adapters ( #6573 )
...
---------
Co-authored-by: yiyixuxu <yixu310@gmail,com>
Co-authored-by: Alvaro Somoza <somoza.alvaro@gmail.com >
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
2024-12-23 13:02:06 +05:30
Sayak Paul
9017578d88
remove omegaconf from the residues 👋 ( #6600 )
...
remove omegaconf 👋
2024-12-23 13:02:06 +05:30
JuanCarlosPi
c9ffa6cd90
Change in ip-adapter docs. CLIPVisionModelWithProjection should be im… ( #6597 )
...
Change in ip-adapter docs. CLIPVisionModelWithProjection should be imported from transformers, not diffusers
2024-12-23 13:02:06 +05:30
Patrick von Platen
3654809062
[Docs] Fix controlnet indent ( #6578 )
2024-12-23 13:02:05 +05:30
Steven Liu
05322fb98d
[docs] mask_blur and padding_mask_crop ( #6498 )
...
new inpaint features
2024-12-23 13:02:05 +05:30
Steven Liu
1a0673574e
[docs] Callbacks ( #6471 )
...
edits
2024-12-23 13:02:05 +05:30
Steven Liu
db484b5494
[docs] Stable video diffusion ( #6472 )
...
svd
2024-12-23 13:02:05 +05:30
Sayak Paul
a8129030a8
minor changes to the SVD doc ( #6466 )
...
minor changes
2024-12-23 13:02:05 +05:30
YiYi Xu
e0faf4b00f
[doc] update inpaint doc to use apply_overlay ( #6364 )
...
add doc
Co-authored-by: yiyixuxu <yixu310@gmail,com>
2024-12-23 13:02:05 +05:30
Sayak Paul
7570fcef40
[Docs] fix: video rendering on svd. ( #6330 )
...
fix: video rendering on svd.
2024-12-23 13:02:05 +05:30
Dhruv Nair
d310bf3589
Interruptable Pipelines ( #5867 )
...
* add interruptable pipelines
* add tests
* updatemsmq
* add interrupt property
* make fix copies
* Revert "make fix copies"
This reverts commit 914b35332b59462357+stevhliu@users.noreply.github.com >
* Update docs/source/en/tutorials/interrupting_diffusion_process.md
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
* update
* fix quality issues
* fix
* update
---------
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
2024-12-23 13:02:05 +05:30
Steven Liu
5433962992
[docs] Batched seeds ( #6237 )
...
batched seed
2023-12-19 16:50:18 -08:00
raven
df476d9f63
[Docs] Fix a code example in the ControlNet Inpainting documentation ( #6236 )
...
fix document on masked image in inpainting controlnet
2023-12-19 13:14:37 -08:00
Fabio Rigano
b65928b556
Add support for IPAdapterFull ( #5911 )
...
* Add support for IPAdapterFull
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
---------
Co-authored-by: YiYi Xu <yixu310@gmail.com >
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
2023-12-07 06:40:39 -10:00
Lucain
75ada25048
Harmonize HF environment variables + deprecate use_auth_token ( #6066 )
...
* Harmonize HF environment variables + deprecate use_auth_token
* fix import
* fix
2023-12-06 22:22:31 +01:00
Steven Liu
b64f835ea7
[docs] Add Kandinsky 3 ( #5988 )
...
* add
* fix api docs
* edits
2023-12-04 10:11:15 -08:00
Steven Liu
6ba4c5395f
[docs] Fix SVD video ( #6004 )
...
Update svd.md
2023-12-01 16:07:47 +01:00
Steven Liu
f72b28c75b
[docs] Fix video link ( #5986 )
...
Update svd.md
2023-11-29 20:52:25 +01:00
Suraj Patil
ada8109d5b
Fix SVD doc ( #5983 )
...
fix url
2023-11-29 19:55:05 +01:00
Patrick von Platen
b34acbdcbc
[SDXL Turbo] Add some docs ( #5982 )
...
* add diffusers example
* add diffusers example
* Comment about making it faster
* Apply suggestions from code review
Co-authored-by: Pedro Cuenca <pedro@huggingface.co >
---------
Co-authored-by: Pedro Cuenca <pedro@huggingface.co >
2023-11-29 19:52:07 +01:00
Suraj Patil
63f767ef15
Add SVD ( #5895 )
...
* begin model
* finish blocks
* add_embedding
* addition_time_embed_dim
* use TimestepEmbedding
* fix temporal res block
* fix time_pos_embed
* fix add_embedding
* add conversion script
* fix model
* up
* add new resnet blocks
* make forward work
* return sample in original shape
* fix temb shape in TemporalResnetBlock
* add spatio temporal transformers
* add vae blocks
* fix blocks
* update
* update
* fix shapes in Alphablender and add time activation in res blcok
* use new blocks
* style
* fix temb shape
* fix SpatioTemporalResBlock
* reuse TemporalBasicTransformerBlock
* fix TemporalBasicTransformerBlock
* use TransformerSpatioTemporalModel
* fix TransformerSpatioTemporalModel
* fix time_context dim
* clean up
* make temb optional
* add blocks
* rename model
* update conversion script
* remove UNetMidBlockSpatioTemporal
* add in init
* remove unused arg
* remove unused arg
* remove more unsed args
* up
* up
* check for None
* update vae
* update up/mid blocks for decoder
* begin pipeline
* adapt scheduler
* add guidance scalings
* fix norm eps in temporal transformers
* add temporal autoencoder
* make pipeline run
* fix frame decodig
* decode in float32
* decode n frames at a time
* pass decoding_t to decode_latents
* fix decode_latents
* vae encode/decode in fp32
* fix dtype in TransformerSpatioTemporalModel
* type image_latents same as image_embeddings
* allow using differnt eps in temporal block for video decoder
* fix default values in vae
* pass num frames in decode
* switch spatial to temporal for mixing in VAE
* fix num frames during split decoding
* cast alpha to sample dtype
* fix attention in MidBlockTemporalDecoder
* fix typo
* fix guidance_scales dtype
* fix missing activation in TemporalDecoder
* skip_post_quant_conv
* add vae conversion
* style
* take guidance scale as input
* up
* allow passing PIL to export_video
* accept fps as arg
* add pipeline and vae in init
* remove hack
* use AutoencoderKLTemporalDecoder
* don't scale image latents
* add unet tests
* clean up unet
* clean TransformerSpatioTemporalModel
* add slow svd test
* clean up
* make temb optional in Decoder mid block
* fix norm eps in TransformerSpatioTemporalModel
* clean up temp decoder
* clean up
* clean up
* use c_noise values for timesteps
* use math for log
* update
* fix copies
* doc
* upcast vae
* update forward pass for gradient checkpointing
* make added_time_ids is tensor
* up
* fix upcasting
* remove post quant conv
* add _resize_with_antialiasing
* fix _compute_padding
* cleanup model
* more cleanup
* more cleanup
* more cleanup
* remove freeu
* remove attn slice
* small clean
* up
* up
* remove extra step kwargs
* remove eta
* remove dropout
* remove callback
* remove merge factor args
* clean
* clean up
* move to dedicated folder
* remove attention_head_dim
* docstr and small fix
* update unet doc strings
* rename decoding_t
* correct linting
* store c_skip and c_out
* cleanup
* clean TemporalResnetBlock
* more cleanup
* clean up vae
* clean up
* begin doc
* more cleanup
* up
* up
* doc
* Improve
* better naming
* better naming
* better naming
* better naming
* better naming
* better naming
* better naming
* better naming
* Apply suggestions from code review
* Default chunk size to None
* add example
* Better
* Apply suggestions from code review
* update doc
* Update src/diffusers/pipelines/stable_diffusion_video/pipeline_stable_diffusion_video.py
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
* style
* Get torch compile working
* up
* rename
* fix doc
* add chunking
* torch compile
* torch compile
* add modelling outputs
* torch compile
* Improve chunking
* Apply suggestions from code review
* Update docs/source/en/using-diffusers/svd.md
* Close diff tag
* remove slicing
* resnet docstr
* add docstr in resnet
* rename
* Apply suggestions from code review
* update tests
* Fix output type latents
* fix more
* fix more
* Update docs/source/en/using-diffusers/svd.md
* fix more
* add pipeline tests
* remove unused arg
* clean up
* make sure get_scaling receives tensors
* fix euler scheduler
* fix get_scalings
* simply euler for now
* remove old test file
* use randn_tensor to create noise
* fix device for rand tensor
* increase expected_max_difference
* fix test_inference_batch_single_identical
* actually fix test_inference_batch_single_identical
* disable test_save_load_float16
* skip test_float16_inference
* skip test_inference_batch_single_identical
* fix test_xformers_attention_forwardGenerator_pass
* Apply suggestions from code review
* update StableVideoDiffusionPipelineSlowTests
* update image
* add diffusers example
* fix more
---------
Co-authored-by: Dhruv Nair <dhruv.nair@gmail.com >
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
Co-authored-by: apolinário <joaopaulo.passos@gmail.com >
2023-11-29 19:13:36 +01:00
Steven Liu
50a749e909
[docs] Fix space ( #5898 )
...
* fix
* minor edits
2023-11-27 11:50:59 -08:00
