Sayak Paul
8373a61a95
[Tests] mark UNetControlNetXSModelTests::test_forward_no_control to be flaky ( #7771 )
...
decorate UNetControlNetXSModelTests::test_forward_no_control with is_flaky
2024-12-23 13:02:11 +05:30
Sayak Paul
b4d6d464d0
[PixArt] fix small nits in pixart sigma ( #7767 )
...
fix small nits in pixart sigma
2024-12-23 13:02:11 +05:30
Junsong Chen
55d7ad64ea
PixArt-Sigma Implementation ( #7654 )
...
* support PixArt-DMD
---------
Co-authored-by: jschen <chenjunsong4@h-partners.com >
Co-authored-by: badayvedat <badayvedat@gmail.com >
Co-authored-by: Vedat Baday <54285744+badayvedat@users.noreply.github.com >
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
Co-authored-by: YiYi Xu <yixu310@gmail.com >
Co-authored-by: yiyixuxu <yixu310@gmail,com>
2024-12-23 13:02:11 +05:30
Dhruv Nair
9de95b0b96
Fix test for consistency decoder. ( #7746 )
...
update
2024-12-23 13:02:11 +05:30
Dhruv Nair
77d3449200
Fix failing VAE tiling test ( #7747 )
...
update
2024-12-23 13:02:11 +05:30
Steven Liu
ed20a5ac49
[docs] Clean up toctree ( #7715 )
...
* toctree
* optim
* feedback
* improve overview
2024-12-23 13:02:11 +05:30
Sayak Paul
9bef15a567
[Metadat utils] fix: json lines ordering. ( #7744 )
...
fix: json lines ordering.
2024-12-23 13:02:11 +05:30
Sai-Suraj-27
5e66907fda
fix: Fixed a wrong decorator by modifying it to @classmethod ( #7653 )
...
* Fixed wrong decorator by modifying it to @classmethod.
* Updated the method and it's argument.
---------
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
2024-12-23 13:02:10 +05:30
Fabio Rigano
030b9de0df
Restore AttnProcessor2_0 in unload_ip_adapter ( #7727 )
...
* Restore AttnProcessor2_0 in unload_ip_adapter
* Fix style
* Update test
---------
Co-authored-by: YiYi Xu <yixu310@gmail.com >
2024-12-23 13:02:10 +05:30
Jenyuan-Huang
4ec19bc3d5
Support InstantStyle ( #7668 )
...
* enable control ip-adapter per-transformer block on-the-fly
---------
Co-authored-by: sayakpaul <spsayakpaul@gmail.com >
Co-authored-by: ResearcherXman <xhs.research@gmail.com >
Co-authored-by: YiYi Xu <yixu310@gmail.com >
2024-12-23 13:02:10 +05:30
Phil Butler
26f6b4c596
Remove redundant lines ( #7396 )
...
Co-authored-by: Dhruv Nair <dhruv.nair@gmail.com >
Co-authored-by: YiYi Xu <yixu310@gmail.com >
2024-12-23 13:02:10 +05:30
Steven Liu
1cce2c1c25
[docs] AutoPipeline ( #7714 )
...
* autopipeline
* edits
* feedback
---------
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
2024-12-23 13:02:10 +05:30
Dhruv Nair
4b518b4a6e
Fix Kandinksy V22 tests ( #7699 )
...
update
2024-12-23 13:02:10 +05:30
Dhruv Nair
6e888b0a69
Update Wuerschten Test ( #7700 )
...
update
2024-12-23 13:02:10 +05:30
Abhinav Gopal
024fb640e0
Animatediff Controlnet Community Pipeline IP Adapter Fix ( #7413 )
...
* fixed encode_image function signature in controlnet animatediff
* copied encode_image from stable diffusion pipeline
---------
Co-authored-by: YiYi Xu <yixu310@gmail.com >
2024-12-23 13:02:10 +05:30
Guillaume LEGENDRE
3cf49fd38f
fix/add tailscale key in case of failure ( #7719 )
...
add tailscale key in case of failure
2024-12-23 13:02:10 +05:30
Sai-Suraj-27
20b5919ab2
fix: Fixed type annotations for compatability with python 3.8 ( #7648 )
...
* Fixed type annotations for compatability with python 3.8
* Add required imports.
2024-12-23 13:02:10 +05:30
Dhruv Nair
eb75ed9110
Cleanup ControlnetXS ( #7701 )
...
* update
* update
2024-12-23 13:02:10 +05:30
Dhruv Nair
5f5c806d9c
Cast height, width to int inside prepare latents ( #7691 )
...
update
2024-12-23 13:02:10 +05:30
YiYi Xu
cd4743d978
adding back test_conversion_when_using_device_map ( #7704 )
...
* style
* Fix device map nits (#7705 )
---------
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
2024-12-23 13:02:10 +05:30
Fabio Rigano
c5a2b97bff
Move IP Adapter Face ID to core ( #7186 )
...
* Switch to peft and multi proj layers
* Move Face ID loading and inference to core
---------
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
2024-12-23 13:02:10 +05:30
Guillaume LEGENDRE
bb07c28442
Add tailscale action to push_test ( #7709 )
2024-12-23 13:02:10 +05:30
Steven Liu
2bed2f4c45
[docs] Pipeline loading ( #7684 )
...
* pipelines
* schedulers and models
* community pipelines
* feedback
2024-12-23 13:02:10 +05:30
Wentian
ff3e9587d9
[Docs] Update TGATE in section optimization. ( #7698 )
...
Update tgate.md
2024-12-23 13:02:10 +05:30
Sayak Paul
d01453e437
[Workflows] remove installation of redundant modules from flax PR tests ( #7662 )
...
remove installation of redundant modules from flax PR tests
2024-12-23 13:02:10 +05:30
Dhruv Nair
d166f496b6
Don't install PEFT with UV in slow tests ( #7697 )
...
* update
* update
2024-12-23 13:02:10 +05:30
Sai-Suraj-27
2b16db59f1
fix: Updated ruff configuration to avoid deprecated configuration warning ( #7637 )
...
Updated ruff configuration to avoid depreceated config.
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
2024-12-23 13:02:10 +05:30
Wentian
92125c6887
[Docs] Add TGATE in section optimization ( #7639 )
...
* Create tgate.md
* Update _toctree.yml
* Update docs/source/en/optimization/tgate.md
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
* Update docs/source/en/optimization/tgate.md
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
* Update docs/source/en/optimization/tgate.md
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
* Update docs/source/en/optimization/tgate.md
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
* Update docs/source/en/optimization/tgate.md
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
* Update docs/source/en/optimization/tgate.md
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
* Update docs/source/en/optimization/tgate.md
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
* Update docs/source/en/optimization/tgate.md
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
* Update docs/source/en/optimization/tgate.md
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
* Update docs/source/en/optimization/tgate.md
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
* Update tgate.md
* Update tgate.md
---------
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
2024-12-23 13:02:10 +05:30
Sayak Paul
4e28bea1bc
[Core] is_cosxl_edit arg in SDXL ip2p. ( #7650 )
...
* is_cosxl_edit arg in SDXL ip2p.
* Empty-Commit
Co-authored-by: Yiyi Xu <yixu310@gmail.com >
* doc
* remove redundant logic.
* reflect drhuv's comments.
---------
Co-authored-by: Yiyi Xu <yixu310@gmail.com >
Co-authored-by: Dhruv Nair <dhruv.nair@gmail.com >
2024-12-23 13:02:10 +05:30
UmerHA
d7895dfa9d
Fixing implementation of ControlNet-XS ( #6772 )
...
* CheckIn - created DownSubBlocks
* Added extra channels, implemented subblock fwd
* Fixed connection sizes
* checkin
* Removed iter, next in forward
* Models for SD21 & SDXL run through
* Added back pipelines, cleared up connections
* Cleaned up connection creation
* added debug logs
* updated logs
* logs: added input loading
* Update umer_debug_logger.py
* log: Loading hint
* Update umer_debug_logger.py
* added logs
* Changed debug logging
* debug: added more logs
* Fixed num_norm_groups
* Debug: Logging all of SDXL input
* Update umer_debug_logger.py
* debug: updated logs
* checkim
* Readded tests
* Removed debug logs
* Fixed Slow Tests
* Added value ckecks | Updated model_cpu_offload_seq
* accelerate-offloading works ; fast tests work
* Made unet & addon explicit in controlnet
* Updated slow tests
* Added dtype/device to ControlNetXS
* Filled in test model paths
* Added image_encoder/feature_extractor to XL pipe
* Fixed fast tests
* Added comments and docstrings
* Fixed copies
* Added docs ; Updates slow tests
* Moved changes to UNetMidBlock2DCrossAttn
* tiny cleanups
* Removed stray prints
* Removed ip adapters + freeU
- Removed ip adapters + freeU as they don't make sense for ControlNet-XS
- Fixed imports of UNet components
* Fixed test_save_load_float16
* Make style, quality, fix-copies
* Changed loading/saving API for ControlNetXS
- Changed loading/saving API for ControlNetXS
- other small fixes
* Removed ControlNet-XS from research examples
* Make style, quality, fix-copies
* Small fixes
- deleted ControlNetXSModel.init_original
- added time_embedding_mix to StableDiffusionControlNetXSPipeline .from_pretrained / StableDiffusionXLControlNetXSPipeline.from_pretrained
- fixed copy hints
* checkin May 11 '23
* CheckIn Mar 12 '24
* Fixed tests for SD
* Added tests for UNetControlNetXSModel
* Fixed SDXL tests
* cleanup
* Delete Pipfile
* CheckIn Mar 20
Started replacing sub blocks by `ControlNetXSCrossAttnDownBlock2D` and `ControlNetXSCrossAttnUplock2D`
* check-in Mar 23
* checkin 24 Mar
* Created init for UNetCnxs and CnxsAddon
* CheckIn
* Made from_modules, from_unet and no_control work
* make style,quality,fix-copies & small changes
* Fixed freezing
* Added gradient ckpt'ing; fixed tests
* Fix slow tests(+compile) ; clear naming confusion
* Don't create UNet in init ; removed class_emb
* Incorporated review feedback
- Deleted get_base_pipeline / get_controlnet_addon for pipes
- Pipes inherit from StableDiffusionXLPipeline
- Made module dicts for cnxs-addon's down/mid/up classes
- Added support for qkv fusion and freeU
* Make style, quality, fix-copies
* Implemented review feedback
* Removed compatibility check for vae/ctrl embedding
* make style, quality, fix-copies
* Delete Pipfile
* Integrated review feedback
- Importing ControlNetConditioningEmbedding now
- get_down/mid/up_block_addon now outside class
- renamed `do_control` to `apply_control`
* Reduced size of test tensors
For this, added `norm_num_groups` as parameter everywhere
* Renamed cnxs-`Addon` to cnxs-`Adapter`
- `ControlNetXSAddon` -> `ControlNetXSAdapter`
- `ControlNetXSAddonDownBlockComponents` -> `DownBlockControlNetXSAdapter`, and similarly for mid/up
- `get_mid_block_addon` -> `get_mid_block_adapter`, and similarly for mid/up
* Fixed save_pretrained/from_pretrained bug
* Removed redundant code
---------
Co-authored-by: Dhruv Nair <dhruv.nair@gmail.com >
2024-12-23 13:02:10 +05:30
Sayak Paul
5301343fc9
don't install peft from the source with uv for now. ( #7679 )
2024-12-23 13:02:10 +05:30
Sayak Paul
a761390f12
fix: metadata token ( #7631 )
2024-12-23 13:02:10 +05:30
Sayak Paul
ab34f394df
make docker-buildx mandatory. ( #7652 )
2024-12-23 13:02:10 +05:30
kabachuha
679d7231e7
Add (Scheduled) Pseudo-Huber Loss training scripts to research projects ( #7527 )
...
* add scheduled pseudo-huber loss training scripts
See #7488
* add reduction modes to huber loss
* [DB Lora] *2 multiplier to huber loss cause of 1/2 a^2 conv.
pairing of c6495def1fdd22958caa19a834c3ab47fb1a6854spsayakpaul@gmail.com >
2024-12-23 13:02:10 +05:30
Benjamin Bossan
a0662fdb27
FIX Setting device for DoRA parameters ( #7655 )
...
Fix a bug that causes the the call to set_lora_device to ignore the DoRA
parameters.
2024-12-23 13:02:10 +05:30
Sai-Suraj-27
499d96bfd1
fix: Replaced deprecated logger.warn with logger.warning ( #7643 )
...
Fixed deprecated logger.warn with logger.warning.
2024-12-23 13:02:10 +05:30
Yiqin Zhao
d47de39a46
Fixed YAML loading. ( #7579 )
2024-12-23 13:02:10 +05:30
dg845
6edad9389c
LCM Distill Scripts Fix Bug when Initializing Target U-Net ( #6848 )
...
* Initialize target_unet from unet rather than teacher_unet so that we correctly add time_embedding.cond_proj if necessary.
* Use UNet2DConditionModel.from_config to initialize target_unet from unet's config.
---------
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
2024-12-23 13:02:10 +05:30
Steven Munn
9fffbee44f
Skip PEFT LoRA Scaling if the scale is 1.0 ( #7576 )
...
* Skip scaling if scale is identity
* move check for weight one to scale and unscale lora
* fix code style/quality
* Empty-Commit
---------
Co-authored-by: Steven Munn <stevenjmunn@gmail.com >
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
Co-authored-by: Steven Munn <5297082+stevenjlm@users.noreply.github.com >
2024-12-23 13:02:10 +05:30
Sayak Paul
42dab12780
[Core] fix img2img pipeline for Playground ( #7627 )
...
* playground vae encoding should use std and mean of the vae.
* style.
* fix-copies.
2024-12-23 13:02:10 +05:30
YiYi Xu
4f08858edb
Fix cpu offload related slow tests ( #7618 )
...
* fix
* up
---------
Co-authored-by: yiyixuxu <yixu310@gmail,com>
2024-12-23 13:02:10 +05:30
Steven Liu
9b5e666e73
[docs] T2I ( #7623 )
...
* refactor t2i
* add code snippets
2024-12-23 13:02:10 +05:30
Steven Liu
f32af25416
[docs] Prompt enhancer ( #7565 )
...
* prompt enhance
* edits
* align titles
* feedback
* feedback
* feedback
* link to style
2024-12-23 13:02:10 +05:30
Sayak Paul
b0849a23fe
[Tests] reduce the model sizes in the SD fast tests ( #7580 )
...
* give it a shot.
* print.
* correct assertion.
* gather results from the rest of the tests.
* change the assertion values where needed.
* remove print statements.
2024-12-23 13:02:10 +05:30
satani99
da8d2fb766
Modularize instruct_pix2pix SD inferencing during and after training in examples ( #7603 )
...
* Modularize instruct_pix2pix code
* quality check
* quality check
---------
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
2024-12-23 13:02:10 +05:30
Sayak Paul
4dc77c29e6
[docs] remove duplicate tip block. ( #7625 )
...
remove duplicate tip block.
2024-12-23 13:02:10 +05:30
IDKiro
1bdd0d0bf6
add the option of upsample function for tiny vae ( #7604 )
...
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
Co-authored-by: YiYi Xu <yixu310@gmail.com >
2024-12-23 13:02:10 +05:30
Sayak Paul
b5585c875d
[Core] add "balanced" device_map support to pipelines ( #6857 )
...
* get device <-> component mapping when using multiple gpus.
* condition the device_map bits.
* relax condition
* device_map progress.
* device_map enhancement
* some cleaning up and debugging
* Apply suggestions from code review
Co-authored-by: Marc Sun <57196510+SunMarc@users.noreply.github.com >
* incorporate suggestions from PR.
* remove multi-gpu condition for now.
* guard check the component -> device mapping
* fix: device_memory variable
* dispatching transformers model to have force_hooks=True
* better guarding for transformers device_map
* introduce support balanced_low_memory and balanced_ultra_low_memory.
* remove device_map patch.
* fix: intermediate variable scoping.
* fix: condition in cpu offload.
* fix: flax class restrictions.
* remove modifications from cpu_offload and model_offload
* incorporate changes.
* add a simple forward pass test
* add: torch_device in get_inputs()
* add: tests
* remove print
* safe-guard to(), model offloading and cpu offloading when balanced is used as a device_map.
* style
* remove .
* safeguard device_map with more checks and remove invalid device_mapping strategues.
* make a class attribute and adjust tests accordingly.
* fix device_map check
* fix test
* adjust comment
* fix: device_map attribute
* fix: dispatching.
* max_memory test for pipeline
* version guard the tests
* fix guard.
* address review feedback.
* reset_device_map method.
* add: test for reset_hf_device_map
* fix a couple things.
* add reset_device_map() in the error message.
* add tests for checking reset_device_map doesn't have unintended consequences.
* fix reset_device_map and offloading tests.
* create _get_final_device_map utility.
* hf_device_map -> _hf_device_map
* add documentation
* add notes suggested by Marc.
* styling.
* Apply suggestions from code review
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
Co-authored-by: Pedro Cuenca <pedro@huggingface.co >
* move updates within gpu condition.
* other docs related things
* note on ignore a device not specified in .
* provide a suggestion if device mapping errors out.
* fix: typo.
* _hf_device_map -> hf_device_map
* Empty-Commit
* add: example hf_device_map.
---------
Co-authored-by: Marc Sun <57196510+SunMarc@users.noreply.github.com >
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
Co-authored-by: Pedro Cuenca <pedro@huggingface.co >
2024-12-23 13:02:10 +05:30
Sayak Paul
d6bed5cdf9
[Workflows] remove installation of libsndfile1-dev and libgl1 from workflows ( #7543 )
...
* remove libsndfile1-dev and libgl1 from workflows and ensure that re present in the respective dockerfiles.
* change to self-hosted runner; let's see 🤞
* add libsndfile1-dev libgl1 for now
* use self-hosted runners for building and push too.
2024-12-23 13:02:10 +05:30
Sayak Paul
35eaa65aa0
[Core] refactor transformer_2d forward logic into meaningful conditions. ( #7489 )
...
* refactor transformer_2d forward logic into meaningful conditions.
* Empty-Commit
* fix: _operate_on_patched_inputs
* fix: _operate_on_patched_inputs
* check
* fix: patch output computation block.
* fix: _operate_on_patched_inputs.
* remove print.
* move operations to blocks.
* more readability neats.
* empty commit
* Apply suggestions from code review
Co-authored-by: Dhruv Nair <dhruv.nair@gmail.com >
* Revert "Apply suggestions from code review"
This reverts commit 12178b1aa0dhruv.nair@gmail.com >
2024-12-23 13:02:10 +05:30
