LLM/diffusers
1
0
Fork 0
mirror of https://github.com/huggingface/diffusers.git synced 2026-02-14 14:55:26 +08:00
Code Issues Packages Projects Releases Wiki Activity
4,913 Commits 743 Branches 94 Tags
4c5082ca685d1dd0a24907890da170e258e5cd01
Commit Graph

4913 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Álvaro Somoza
cfb49c5efe [Official callbacks] SDXL Controlnet CFG Cutoff (#9311) 
* initial proposal

* style
 2024-12-23 13:02:17 +05:30
Fanli Lin
538040ca32 fix bug in require_accelerate_version_greater (#9746) 
fix bug
 2024-12-23 13:02:17 +05:30
Sayak Paul
23cf712349 post-release 0.31.0 (#9742) 
* post-release

* style
 2024-12-23 13:02:17 +05:30
Dhruv Nair
12799075a0 is_safetensors_compatible fix (#9741) 
update
 2024-12-23 13:02:17 +05:30
Dhruv Nair
11c27c87f0 Fix typos (#9739) 
* update

* update

* update

* update

* update

* update
 2024-12-23 13:02:17 +05:30
Sayak Paul
a2591a6a28 [bitsandbbytes] follow-ups (#9730) 
* bnb follow ups.

* add a warning when dtypes mismatch.

* fx-copies

* clear cache.

* check_if_quantized_param

* add a check on shape.

* updates

* docs

* improve readability.

* resources.

* fix
 2024-12-23 13:02:17 +05:30
Álvaro Somoza
797e22f0f7 [Fix] Using sharded checkpoints with gated repositories (#9737) 
fix
 2024-12-23 13:02:17 +05:30
Yu Zheng
71317a8383 Update sd3 controlnet example (#9735) 
* use make_image_grid in diffusers.utils

* use checkpoint on the Hub
 2024-12-23 13:02:17 +05:30
Tolga Cangöz
fdc716ffa6 Fix schedule_shifted_power usage in 🪆Matryoshka Diffusion Models (#9723) 
* [matryoshka.py] Add schedule_shifted_power attribute and update get_schedule_shifted method
 2024-12-23 13:02:17 +05:30
YiYi Xu
1aedab7311 fix singlestep dpm tests (#9716) 
fix

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
 2024-12-23 13:02:17 +05:30
G.O.D
1051f2179a [bugfix] reduce float value error when adding noise (#9004) 
* Update train_controlnet.py

reduce float value error for bfloat16

* Update train_controlnet_sdxl.py

* style

---------

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
Co-authored-by: yiyixuxu <yixu310@gmail.com>
 2024-12-23 13:02:17 +05:30
YiYi Xu
0e1e923a88 minor doc/test update (#9734) 
* update some docs and tests!

---------

Co-authored-by: Aryan <contact.aryanvs@gmail.com>
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
Co-authored-by: Aryan <aryan@huggingface.co>
Co-authored-by: apolinário <joaopaulo.passos@gmail.com>
 2024-12-23 13:02:17 +05:30
timdalxx
898a5e4fe9 [docs] add docstrings in pipline_stable_diffusion.py (#9590) 
* fix the issue on flux dreambooth lora training

* update : origin main code

* docs: update pipeline_stable_diffusion docstring

* docs: update pipeline_stable_diffusion docstring

* Update src/diffusers/pipelines/stable_diffusion/pipeline_stable_diffusion.py

Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>

* fix: style

* fix: style

* fix: copies

* make fix-copies

* remove extra newline

---------

Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
Co-authored-by: Aryan <aryan@huggingface.co>
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
 2024-12-23 13:02:17 +05:30
Sayak Paul
8faf632fbc [Docs] docs to xlabs controlnets. (#9688) 
* docs to xlabs controlnets.

Co-authored-by: Anzhella Pankratova <son0shad@gmail.com>

* Apply suggestions from code review

Co-authored-by: Anzhella Pankratova <54744846+Anghellia@users.noreply.github.com>

---------

Co-authored-by: Anzhella Pankratova <son0shad@gmail.com>
Co-authored-by: Anzhella Pankratova <54744846+Anghellia@users.noreply.github.com>
 2024-12-23 13:02:17 +05:30
Chenyu Li
5641263256 Fix typo in cogvideo pipeline (#9722) 
Fix type in cogvideo pipeline
 2024-12-23 13:02:17 +05:30
Sayak Paul
52d659b92c [Quantization] Add quantization support for bitsandbytes (#9213) 
* quantization config.

* fix-copies

* fix

* modules_to_not_convert

* add bitsandbytes utilities.

* make progress.

* fixes

* quality

* up

* up

rotary embedding refactor 2: update comments, fix dtype for use_real=False (#9312)

fix notes and dtype

up

up

* minor

* up

* up

* fix

* provide credits where due.

* make configurations work.

* fixes

* fix

* update_missing_keys

* fix

* fix

* make it work.

* fix

* provide credits to transformers.

* empty commit

* handle to() better.

* tests

* change to bnb from bitsandbytes

* fix tests

fix slow quality tests

SD3 remark

fix

complete int4 tests

add a readme to the test files.

add model cpu offload tests

warning test

* better safeguard.

* change merging status

* courtesy to transformers.

* move  upper.

* better

* make the unused kwargs warning friendlier.

* harmonize changes with https://github.com/huggingface/transformers/pull/33122

* style

* trainin tests

* feedback part i.

* Add Flux inpainting and Flux Img2Img (#9135)

---------

Co-authored-by: yiyixuxu <yixu310@gmail.com>

Update `UNet2DConditionModel`'s error messages (#9230)

* refactor

[CI] Update Single file Nightly Tests (#9357)

* update

* update

feedback.

improve README for flux dreambooth lora (#9290)

* improve readme

* improve readme

* improve readme

* improve readme

fix one uncaught deprecation warning for accessing vae_latent_channels in VaeImagePreprocessor (#9372)

deprecation warning vae_latent_channels

add mixed int8 tests and more tests to nf4.

[core] Freenoise memory improvements (#9262)

* update

* implement prompt interpolation

* make style

* resnet memory optimizations

* more memory optimizations; todo: refactor

* update

* update animatediff controlnet with latest changes

* refactor chunked inference changes

* remove print statements

* update

* chunk -> split

* remove changes from incorrect conflict resolution

* remove changes from incorrect conflict resolution

* add explanation of SplitInferenceModule

* update docs

* Revert "update docs"

This reverts commit c55a50a271.

* update docstring for freenoise split inference

* apply suggestions from review

* add tests

* apply suggestions from review

quantization docs.

docs.

* Revert "Add Flux inpainting and Flux Img2Img (#9135)"

This reverts commit 5799954dd4.

* tests

* don

* Apply suggestions from code review

Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>

* contribution guide.

* changes

* empty

* fix tests

* harmonize with https://github.com/huggingface/transformers/pull/33546.

* numpy_cosine_distance

* config_dict modification.

* remove if config comment.

* note for load_state_dict changes.

* float8 check.

* quantizer.

* raise an error for non-True low_cpu_mem_usage values when using quant.

* low_cpu_mem_usage shenanigans when using fp32 modules.

* don't re-assign _pre_quantization_type.

* make comments clear.

* remove comments.

* handle mixed types better when moving to cpu.

* add tests to check if we're throwing warning rightly.

* better check.

* fix 8bit test_quality.

* handle dtype more robustly.

* better message when keep_in_fp32_modules.

* handle dtype casting.

* fix dtype checks in pipeline.

* fix warning message.

* Update src/diffusers/models/modeling_utils.py

Co-authored-by: YiYi Xu <yixu310@gmail.com>

* mitigate the confusing cpu warning

---------

Co-authored-by: Vishnu V Jaddipal <95531133+Gothos@users.noreply.github.com>
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
Co-authored-by: YiYi Xu <yixu310@gmail.com>
 2024-12-23 13:02:17 +05:30
Aryan
f5664212da make deps_table_update to fix CI tests (#9720) 
* update

* dummy change to trigger CI; will revert

* no deps peft

* np deps

* todo

---------

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
 2024-12-23 13:02:17 +05:30
Sayak Paul
55996cf25f [Docker] pin torch versions in the dockerfiles. (#9721) 
* pin torch versions in the dockerfiles.

* more
 2024-12-23 13:02:17 +05:30
Aryan
3a89338d49 [CI] pin max torch version to fix CI errors (#9709) 
* pin max torch version

* update

* Update setup.py
 2024-12-23 13:02:17 +05:30
hlky
ec6bfdf069 Add prompt scheduling callback to community scripts (#9718) 2024-12-23 13:02:17 +05:30
bonlime
a3e79f6e35 Fix bug in Textual Inversion Unloading (#9304) 
* Update textual_inversion.py

* add unload test

* add comment

* fix style

---------

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
Co-authored-by: Your Name <you@example.com>
Co-authored-by: YiYi Xu <yixu310@gmail.com>
 2024-12-23 13:02:17 +05:30
Linoy Tsaban
da0b9e1274 [advanced flux lora script] minor updates to readme (#9705) 
* fix arg naming

* fix arg naming

* fix arg naming

* fix arg naming
 2024-12-23 13:02:17 +05:30
Aryan
afba632dec [refactor] DiffusionPipeline.download (#9557) 
* update

---------

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
Co-authored-by: Dhruv Nair <dhruv.nair@gmail.com>
 2024-12-23 13:02:17 +05:30
Linoy Tsaban
65fe4b98d9 [Flux] Add advanced training script + support textual inversion inference (#9434) 
* add ostris trainer to README & add cache latents of vae

* add ostris trainer to README & add cache latents of vae

* style

* readme

* add test for latent caching

* add ostris noise scheduler
9ee1ef2a0a/toolkit/samplers/custom_flowmatch_sampler.py (L95)

* style

* fix import

* style

* fix tests

* style

* --change upcasting of transformer?

* update readme according to main

* add pivotal tuning for CLIP

* fix imports, encode_prompt call,add TextualInversionLoaderMixin to FluxPipeline for inference

* TextualInversionLoaderMixin support for FluxPipeline for inference

* move changes to advanced flux script, revert canonical

* add latent caching to canonical script

* revert changes to canonical script to keep it separate from https://github.com/huggingface/diffusers/pull/9160

* revert changes to canonical script to keep it separate from https://github.com/huggingface/diffusers/pull/9160

* style

* remove redundant line and change code block placement to align with logic

* add initializer_token arg

* add transformer frac for range support from pure textual inversion to the orig pivotal tuning

* support pure textual inversion - wip

* adjustments to support pure textual inversion and transformer optimization in only part of the epochs

* fix logic when using initializer token

* fix pure_textual_inversion_condition

* fix ti/pivotal loading of last validation run

* remove embeddings loading for ti in final training run (to avoid adding huggingface hub dependency)

* support pivotal for t5

* adapt pivotal for T5 encoder

* adapt pivotal for T5 encoder and support in flux pipeline

* t5 pivotal support + support fo pivotal for clip only or both

* fix param chaining

* fix param chaining

* README first draft

* readme

* readme

* readme

* style

* fix import

* style

* add fix from https://github.com/huggingface/diffusers/pull/9419

* add to readme, change function names

* te lr changes

* readme

* change concept tokens logic

* fix indices

* change arg name

* style

* dummy test

* revert dummy test

* reorder pivoting

* add warning in case the token abstraction is not the instance prompt

* experimental - wip - specific block training

* fix documentation and token abstraction processing

* remove transformer block specification feature (for now)

* style

* fix copies

* fix indexing issue when --initializer_concept has different amounts

* add if TextualInversionLoaderMixin to all flux pipelines

* style

* fix import

* fix imports

* address review comments - remove necessary prints & comments, use pin_memory=True, use free_memory utils, unify warning and prints

* style

* logger info fix

* make lora target modules configurable and change the default

* make lora target modules configurable and change the default

* style

* make lora target modules configurable and change the default, add notes to readme

* style

* add tests

* style

* fix repo id

* add updated requirements for advanced flux

* fix indices of t5 pivotal tuning embeddings

* fix path in test

* remove `pin_memory`

* fix filename of embedding

* fix filename of embedding

---------

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
Co-authored-by: YiYi Xu <yixu310@gmail.com>
 2024-12-23 13:02:17 +05:30
Aryan
f15d9921f1 [tests] fix name and unskip CogI2V integration test (#9683) 
update

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
 2024-12-23 13:02:17 +05:30
Aryan
6796ef0c4f [core] improve VAE encode/decode framewise batching (#9684) 
* update

* apply suggestions from review

---------

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
 2024-12-23 13:02:17 +05:30
Aryan
5f88292f99 [pipeline] CogVideoX-Fun Control (#9671) 
* cogvideox-fun control

* make style

* make fix-copies

* karras schedulers

* Update src/diffusers/pipelines/cogvideo/pipeline_cogvideox_fun_control.py

Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>

* Update docs/source/en/api/pipelines/cogvideox.md

Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>

* apply suggestions from review

---------

Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
 2024-12-23 13:02:17 +05:30
Jongho Choi
6e02cd7241 [peft] simple update when unscale (#9689) 
Update peft_utils.py

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
 2024-12-23 13:02:17 +05:30
Linoy Tsaban
85dde53e30 [SD3 dreambooth-lora training] small updates + bug fixes (#9682) 
* add latent caching + smol updates

* update license

* replace with free_memory

* add --upcast_before_saving to allow saving transformer weights in lower precision

* fix models to accumulate

* fix mixed precision issue as proposed in https://github.com/huggingface/diffusers/pull/9565

* smol update to readme

* style

* fix caching latents

* style

* add tests for latent caching

* style

* fix latent caching

---------

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
 2024-12-23 13:02:17 +05:30
Sayak Paul
674437a39f [LoRA] log a warning when there are missing keys in the LoRA loading. (#9622) 
* log a warning when there are missing keys in the LoRA loading.

* handle missing keys and unexpected keys better.

* add tests

* fix-copies.

* updates

* tests

* concat warning.

* Add Differential Diffusion to Kolors (#9423)

* Added diff diff support for kolors img2img

* Fized relative imports

* Fized relative imports

* Added diff diff support for Kolors

* Fized import issues

* Added map

* Fized import issues

* Fixed naming issues

* Added diffdiff support for Kolors img2img pipeline

* Removed example docstrings

* Added map input

* Updated latents

Co-authored-by: Álvaro Somoza <asomoza@users.noreply.github.com>

* Updated `original_with_noise`

Co-authored-by: Álvaro Somoza <asomoza@users.noreply.github.com>

* Improved code quality

---------

Co-authored-by: Álvaro Somoza <asomoza@users.noreply.github.com>

* FluxMultiControlNetModel (#9647)

* tests

* Update src/diffusers/loaders/lora_pipeline.py

Co-authored-by: YiYi Xu <yixu310@gmail.com>

* fix

---------

Co-authored-by: M Saqlain <118016760+saqlain2204@users.noreply.github.com>
Co-authored-by: Álvaro Somoza <asomoza@users.noreply.github.com>
Co-authored-by: hlky <hlky@hlky.ac>
Co-authored-by: YiYi Xu <yixu310@gmail.com>
 2024-12-23 13:02:17 +05:30
Charchit Sharma
64139fadb4 Resolves [BUG] 'GatheredParameters' object is not callable (#9614) 
* gatherparams bug

* calling context lib object

* fix

---------

Co-authored-by: Aryan <aryan@huggingface.co>
 2024-12-23 13:02:17 +05:30
glide-the
1b7000c836 Docs: CogVideoX (#9578) 
* CogVideoX docs


---------

Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
Co-authored-by: YiYi Xu <yixu310@gmail.com>
 2024-12-23 13:02:17 +05:30
YiYi Xu
dd83a81d65 [authored by @Anghellia) Add support of Xlabs Controlnets #9638 (#9687) 
* Add support of Xlabs Controlnets


---------

Co-authored-by: Anzhella Pankratova <son0shad@gmail.com>
 2024-12-23 13:02:17 +05:30
Aryan
48c973cbbb [training] CogVideoX-I2V LoRA (#9482) 
* update

* update

* update

* update

* update

* add coauthor

Co-Authored-By: yuan-shenghai <963658029@qq.com>

* add coauthor

Co-Authored-By: Shenghai Yuan <140951558+SHYuanBest@users.noreply.github.com>

* update

Co-Authored-By: yuan-shenghai <963658029@qq.com>

* update

---------

Co-authored-by: yuan-shenghai <963658029@qq.com>
Co-authored-by: Shenghai Yuan <140951558+SHYuanBest@users.noreply.github.com>
 2024-12-23 13:02:17 +05:30
Ahnjj_DEV
1b5be33a1d Fix some documentation in ./src/diffusers/models/adapter.py (#9591) 
* Fix some documentation in ./src/diffusers/models/adapter.py

* Update src/diffusers/models/adapter.py

* Update src/diffusers/models/adapter.py

* Update src/diffusers/models/adapter.py

* Update src/diffusers/models/adapter.py

* Update src/diffusers/models/adapter.py

* Update src/diffusers/models/adapter.py

* Update src/diffusers/models/adapter.py

* Update src/diffusers/models/adapter.py

* Update src/diffusers/models/adapter.py

* Update src/diffusers/models/adapter.py

* Update src/diffusers/models/adapter.py

* Update src/diffusers/models/adapter.py

* Update src/diffusers/models/adapter.py

* Update src/diffusers/models/adapter.py

* Update src/diffusers/models/adapter.py

* Update src/diffusers/models/adapter.py

* Update src/diffusers/models/adapter.py

* Update src/diffusers/models/adapter.py

Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>

* Update src/diffusers/models/adapter.py

Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>

* Update src/diffusers/models/adapter.py

Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>

* Update src/diffusers/models/adapter.py

Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>

* Update src/diffusers/models/adapter.py

Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>

* Update src/diffusers/models/adapter.py

Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>

* Update src/diffusers/models/adapter.py

Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>

* run make style

* make style & fix

* make style : 0.1.5 version ruff

* revert changes to examples

---------

Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
Co-authored-by: Aryan <aryan@huggingface.co>
 2024-12-23 13:02:17 +05:30
wony617
b92255af3a [docs] refactoring docstrings in models/embeddings_flax.py (#9592) 
* [docs] refactoring docstrings in `models/embeddings_flax.py`

* Update src/diffusers/models/embeddings_flax.py

* make style

---------

Co-authored-by: Aryan <aryan@huggingface.co>
 2024-12-23 13:02:17 +05:30
wony617
453ababb3f [docs] refactoring docstrings in community/hd_painter.py (#9593) 
* [docs] refactoring docstrings in community/hd_painter.py

* Update examples/community/hd_painter.py

Co-authored-by: Aryan <contact.aryanvs@gmail.com>

* make style

---------

Co-authored-by: Aryan <contact.aryanvs@gmail.com>
Co-authored-by: Aryan <aryan@huggingface.co>
 2024-12-23 13:02:17 +05:30
Jiwook Han
3e941e92a1 [doc] Fix some docstrings in src/diffusers/training_utils.py (#9606) 
* refac: docstrings in training_utils.py

* fix: manual edits

* run make style

* add docstring at cast_training_params

---------

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
 2024-12-23 13:02:17 +05:30
Charchit Sharma
1f3bc5fce8 refactor image_processor.py file (#9608) 
* refactor image_processor file

* changes as requested

* +1 edits

* quality fix

* indent issue

---------

Co-authored-by: Aryan <aryan@huggingface.co>
Co-authored-by: YiYi Xu <yixu310@gmail.com>
 2024-12-23 13:02:17 +05:30
0x名無し
1c7ea5aa29 Dreambooth lora flux bug 3dtensor to 2dtensor (#9653) 
* fixed issue #9350, Tensor is deprecated

* ran make style
 2024-12-23 13:02:17 +05:30
Sayak Paul
c3c6a55bcb [Chore] fix import of EntryNotFoundError. (#9676) 
fix import of EntryNotFoundError.
 2024-12-23 13:02:17 +05:30
Robin
bb20f59d83 [Fix] when run load pretain with local_files_only, local variable 'cached_folder' referenced before assignment (#9376) 
Fix local variable 'cached_folder' referenced before assignment in hub_utils.py

Fix when use `local_files_only=True` with `subfolder`, local variable 'cached_folder' referenced before assignment issue.

Co-authored-by: YiYi Xu <yixu310@gmail.com>
 2024-12-23 13:02:17 +05:30
hlky
d6b2dcfa11 Slight performance improvement to Euler, EDMEuler, FlowMatchHeun, KDPM2Ancestral (#9616) 
* Slight performance improvement to Euler

* Slight performance improvement to EDMEuler

* Slight performance improvement to FlowMatchHeun

* Slight performance improvement to KDPM2Ancestral

* Update KDPM2AncestralDiscreteSchedulerTest

---------

Co-authored-by: YiYi Xu <yixu310@gmail.com>
 2024-12-23 13:02:17 +05:30
hlky
0ed01ee10a Refactor SchedulerOutput and add pred_original_sample in DPMSolverSDE, Heun, KDPM2Ancestral and KDPM2 (#9650) 
Refactor SchedulerOutput and add pred_original_sample

Co-authored-by: YiYi Xu <yixu310@gmail.com>
 2024-12-23 13:02:17 +05:30
hlky
48f795f991 Convert list/tuple of HunyuanDiT2DControlNetModel to HunyuanDiT2DMultiControlNetModel (#9651) 
Convert list/tuple of HunyuanDiT2DControlNetModel to HunyuanDiT2DMultiControlNetModel

Co-authored-by: YiYi Xu <yixu310@gmail.com>
 2024-12-23 13:02:17 +05:30
hlky
5844c7c9d4 Convert list/tuple of SD3ControlNetModel to SD3MultiControlNetModel (#9652) 
Convert list/tuple of SD3ControlNetModel to SD3MultiControlNetModel

Co-authored-by: YiYi Xu <yixu310@gmail.com>
 2024-12-23 13:02:17 +05:30
hlky
be464ed2af Add pred_original_sample to if not return_dict path (#9649) 
Co-authored-by: YiYi Xu <yixu310@gmail.com>
 2024-12-23 13:02:17 +05:30
SahilCarterr
3fdfa1d828 Added Lora Support to SD3 Img2Img Pipeline (#9659) 
* add lora
 2024-12-23 13:02:17 +05:30
Tolga Cangöz
bc2cbfba5d [Community Pipeline] Add 🪆Matryoshka Diffusion Models (#9157) 2024-12-23 13:02:17 +05:30
Leo Jiang
76d9796f25 Improve the performance and suitable for NPU computing (#9642) 
* Improve the performance and suitable for NPU

* Improve the performance and suitable for NPU computing

* Improve the performance and suitable for NPU

* Improve the performance and suitable for NPU

* Improve the performance and suitable for NPU

* Improve the performance and suitable for NPU

---------

Co-authored-by: 蒋硕 <jiangshuo9@h-partners.com>
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
 2024-12-23 13:02:17 +05:30