LLM/diffusers
1
0
Fork 0
mirror of https://github.com/huggingface/diffusers.git synced 2026-04-09 09:17:10 +08:00
Code Issues Packages Projects Releases Wiki Activity
6,384 Commits 804 Branches 96 Tags
modular-doc
Commit Graph

6384 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
yiyi@huggingface.co
1c8bbb5341 up 2026-04-02 20:25:01 +00:00
YiYi Xu
892910648b Update docs/source/en/_toctree.yml 
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
 2026-04-01 11:55:26 -10:00
YiYi Xu
346f8a0e43 Update docs/source/en/modular_diffusers/auto_docstring.md 
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
 2026-04-01 11:55:04 -10:00
YiYi Xu
626f945e68 Update docs/source/en/modular_diffusers/auto_docstring.md 
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
 2026-04-01 11:54:50 -10:00
YiYi Xu
323c08fd67 Update docs/source/en/modular_diffusers/auto_docstring.md 
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
 2026-04-01 11:54:38 -10:00
YiYi Xu
47ab73da89 Update docs/source/en/modular_diffusers/auto_docstring.md 
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
 2026-04-01 11:54:28 -10:00
YiYi Xu
b26a7fa11a Update docs/source/en/modular_diffusers/auto_docstring.md 
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
 2026-04-01 11:54:21 -10:00
YiYi Xu
77837980c7 Update docs/source/en/modular_diffusers/auto_docstring.md 
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
 2026-04-01 11:54:12 -10:00
YiYi Xu
b593c2eb63 Update docs/source/en/modular_diffusers/auto_docstring.md 
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
 2026-04-01 11:54:03 -10:00
YiYi Xu
9379cd3e1f Update docs/source/en/modular_diffusers/auto_docstring.md 
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
 2026-04-01 11:53:53 -10:00
yiyi@huggingface.co
1d2002b705 [docs] add auto docstring and parameter templates documentation for modular diffusers 
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-04-01 20:03:53 +00:00
Steven Liu
e365d749a1 [docs] deprecate pipelines (#13157) 
* deprecate

* fix

* fix

* fix

* fix

* remove deprecated .md files

* update links

* fix
 2026-04-01 10:16:23 -07:00
Andrew Ross
b9353819a4 corrects single file path validation logic (#13363) 
* corrects single file path validation logic

* Update tests/modular_pipelines/test_modular_pipelines_common.py

Co-authored-by: Dhruv Nair <dhruv.nair@gmail.com>

---------

Co-authored-by: Dhruv Nair <dhruv.nair@gmail.com>
 2026-04-01 20:38:42 +05:30
hf-security-analysis[bot]
514bba0696 chore: update claude_review.yml (#13374) 
fix(security): remediate workflow vulnerability in .github/workflows/claude_review.yml

Co-authored-by: hf-security-analysis[bot] <265538906+hf-security-analysis[bot]@users.noreply.github.com>
 2026-04-01 10:18:29 +05:30
YangKai0616
0325ca4c59 Fix MotionConv2d to cast blur_kernel to input dtype instead of reverse (#13364) 
Co-authored-by: dg845 <58458699+dg845@users.noreply.github.com>
 2026-03-31 02:53:12 -07:00
Sayak Paul
a8075425d8 [ci] support claude reviewing on forks. (#13365) 
* support claude reviewing on forks.

* sanitization

* tighten system prompt.

* use latest checkout

* remove id-token
 2026-03-31 14:56:08 +05:30
YangKai0616
b88e60bd1b Fix: ensure consistent dtype and eval mode in pipeline save/load tests (#13339) 
* Fix: ensure consistent dtype and eval mode in pipeline save/load tests

* Modify according to the comments

* Update according to the comments

* Update comment

* Code quality

* cast buffers to torch.float16

* conflict

* Fix

---------

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
 2026-03-31 14:21:28 +05:30
Pranav Thombre
7e463ea4cc [docs] Add NeMo Automodel training guide (#13306) 
* [docs] Add NeMo Automodel training guide

Signed-off-by: Pranav Prashant Thombre <pthombre@nvidia.com>

* Update docs/source/en/training/nemo_automodel.md

Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>

* Update docs/source/en/training/nemo_automodel.md

Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>

* adding contacts into the readme

* Apply suggestion from @stevhliu

Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>

* Apply suggestion from @stevhliu

Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>

* Apply suggestion from @stevhliu

Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>

* Apply suggestion from @stevhliu

Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>

* Apply suggestion from @stevhliu

Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>

* Apply suggestion from @stevhliu

Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>

* Apply suggestion from @stevhliu

Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>

* Apply suggestion from @stevhliu

Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>

* Apply suggestion from @stevhliu

Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>

* Apply suggestion from @stevhliu

Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>

* Apply suggestion from @stevhliu

Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>

* Address CR comments

Signed-off-by: Pranav Prashant Thombre <pthombre@nvidia.com>

* Update docs/source/en/training/nemo_automodel.md

Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>

* Update docs/source/en/training/nemo_automodel.md

Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>

---------

Signed-off-by: Pranav Prashant Thombre <pthombre@nvidia.com>
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
Co-authored-by: linnan wang <wangnan318@gmail.com>
 2026-03-30 10:21:58 -07:00
tcaimm
7f2b34bced Add train flux2 series lora config (#13011) 
* feat(lora): support FLUX.2 single blocks + update README

* add img2img config & add explanatory comments

* simple modify

---------

Co-authored-by: Linoy Tsaban <57615435+linoytsaban@users.noreply.github.com>
 2026-03-30 14:22:04 +03:00
Cheung Ka Wai
e1e7d58a4a Fix Ulysses SP backward with SDPA (#13328) 
* add UT for backward

* fix SDPA attention backward
 2026-03-30 15:15:27 +05:30
Steven Liu
a93f7f137a [docs] refactor model skill (#13334) 
* refactor

* feedback

* feedback

---------

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
 2026-03-29 23:13:52 -07:00
Sayak Paul
10ec3040a2 [ci] move to assert instead of self.Assert* (#13366) 
move to assert instead of self.Assert*
 2026-03-30 11:09:14 +05:30
Howard Zhang
f2be8bd6b3 change minimum version guard for torchao to 0.15.0 (#13355) 2026-03-28 09:11:51 +05:30
Sayak Paul
7da22b9db5 [ci] include checkout step in claude review workflow (#13352) 
up
 2026-03-27 17:28:31 +05:30
Howard Zhang
1fe2125802 remove str option for quantization config in torchao (#13291) 
* remove str option for quantization config in torchao

* Apply style fixes

* minor fixes

* Added AOBaseConfig docs to torchao.md

* minor fixes for removing str option torchao

* minor change to add back int and uint check

* minor fixes

* minor fixes to tests

* Update tests/quantization/torchao/test_torchao.py

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>

* Update docs/source/en/quantization/torchao.md

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>

* Update tests/quantization/torchao/test_torchao.py

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>

* version=2 update to test_torchao.py

---------

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
 2026-03-27 08:52:37 +05:30
dg845
7298f5be93 Update LTX-2 Docs to Cover LTX-2.3 Models (#13337) 
* Update LTX-2 docs to cover multimodal guidance and prompt enhancement

* Apply suggestions from code review

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>

* Apply reviewer feedback

---------

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
 2026-03-26 17:51:29 -07:00
Sayak Paul
b757035df6 fix claude workflow to include id-token with write. (#13338) 2026-03-26 15:39:10 +05:30
kaixuanliu
41e1003316 avoid hardcode device in flux-control example (#13336) 
Signed-off-by: Liu, Kaixuan <kaixuan.liu@intel.com>
 2026-03-26 12:40:53 +05:30
Sayak Paul
85ffcf1db2 [tests] Tests for conditional pipeline blocks (#13247) 
* implement test suite for conditional blocks.

* remove

* another fix.

* Revert "another fix."

This reverts commit ab07b603ab.
 2026-03-26 08:48:16 +05:30
Steven Liu
cbf4d9a3c3 [docs] kernels (#13139) 
* kernels

* feedback
 2026-03-25 09:31:54 -07:00
Sayak Paul
426daabad9 [ci] claude in ci. (#13297) 
* claude in ci.

* review feedback.
 2026-03-25 21:30:06 +05:30
Kashif Rasul
762ae059fa [LLADA2] documentation fixes (#13333) 
documentation fixes
 2026-03-25 17:49:31 +05:30
Kashif Rasul
5d207e756e [Discrete Diffusion] Add LLaDA2 pipeline (#13226) 
* feat: add LLaDA2 and BlockRefinement pipelines for discrete text diffusion

Add support for LLaDA2/LLaDA2.1 discrete diffusion text generation:
- BlockRefinementPipeline: block-wise iterative refinement with confidence-based
  token commitment, supporting editing threshold for LLaDA2.1 models
- LLaDA2Pipeline: convenience wrapper with LLaDA2-specific defaults
- DiscreteDiffusionPipelineMixin: shared SAR sampling utilities (top-k, top-p,
  temperature) and prompt/prefix helpers
- compute_confidence_aware_loss: CAP-style training loss
- Examples: sampling scripts for LLaDA2 and block refinement, training scripts
  with Qwen causal LM
- Docs and tests included

* feat: add BlockRefinementScheduler for commit-by-confidence scheduling

Extract the confidence-based token commit logic from BlockRefinementPipeline
into a dedicated BlockRefinementScheduler, following diffusers conventions.

The scheduler owns:
- Transfer schedule computation (get_num_transfer_tokens)
- Timestep management (set_timesteps)
- Step logic: confidence-based mask-filling and optional token editing

The pipeline now delegates scheduling to self.scheduler.step() and accepts
a scheduler parameter in __init__.

* test: add unit tests for BlockRefinementScheduler

12 tests covering set_timesteps, get_num_transfer_tokens, step logic
(confidence-based commits, threshold behavior, editing, prompt masking,
batched inputs, tuple output).

* docs: add toctree entries and standalone scheduler doc page

- Add BlockRefinement and LLaDA2 to docs sidebar navigation
- Add BlockRefinementScheduler to schedulers sidebar navigation
- Move scheduler autodoc to its own page under api/schedulers/

* feat: add --revision flag and fix dtype deprecation in sample_llada2.py

- Add --revision argument for loading model revisions from the Hub
- Replace deprecated torch_dtype with dtype for transformers 5.x compat

* fix: use 1/0 attention mask instead of 0/-inf for LLaDA2 compat

LLaDA2 models expect a boolean-style (1/0) attention mask, not an
additive (0/-inf) mask. The model internally converts to additive,
so passing 0/-inf caused double-masking and gibberish output.

* refactor: consolidate training scripts into single train_block_refinement.py

- Remove toy train_block_refinement_cap.py (self-contained demo with tiny model)
- Rename train_block_refinement_qwen_cap.py to train_block_refinement.py
  (already works with any causal LM via AutoModelForCausalLM)
- Fix torch_dtype deprecation and update README with correct script names

* fix formatting

* docs: improve LLaDA2 and BlockRefinement documentation

- Add usage examples with real model IDs and working code
- Add recommended parameters table for LLaDA2.1 quality/speed modes
- Note that editing is LLaDA2.1-only (not for LLaDA2.0 models)
- Remove misleading config defaults section from BlockRefinement docs

* feat: set LLaDA2Pipeline defaults to recommended model parameters

- threshold: 0.95 -> 0.7 (quality mode)
- max_post_steps: 0 -> 16 (recommended for LLaDA2.1, harmless for 2.0)
- eos_early_stop: False -> True (stop at EOS token)

block_length=32, steps=32, temperature=0.0 were already correct.
editing_threshold remains None (users enable for LLaDA2.1 models).

* feat: default editing_threshold=0.5 for LLaDA2.1 quality mode

LLaDA2.1 is the current generation. Users with LLaDA2.0 models can
disable editing by passing editing_threshold=None.

* fix: align sampling utilities with official LLaDA2 implementation

- top_p filtering: add shift-right to preserve at least one token above
  threshold (matches official code line 1210)
- temperature ordering: apply scaling before top-k/top-p filtering so
  filtering operates on scaled logits (matches official code lines 1232-1235)
- greedy branch: return argmax directly when temperature=0 without
  filtering (matches official code lines 1226-1230)

* refactor: remove duplicate prompt encoding, reuse mixin's _prepare_input_ids

LLaDA2Pipeline._prepare_prompt_ids was a near-copy of
DiscreteDiffusionPipelineMixin._prepare_input_ids. Remove the duplicate
and call the mixin method directly. Also simplify _extract_input_ids
since we always pass return_dict=True.

* formatting

* fix: replace deprecated torch_dtype with dtype in examples and docstrings

- Update EXAMPLE_DOC_STRING to use dtype= and LLaDA2.1-mini model ID
- Fix sample_block_refinement.py to use dtype=

* remove BlockRefinementPipeline

* cleanup

* fix readme

* Update src/diffusers/pipelines/llada2/pipeline_llada2.py

Co-authored-by: YiYi Xu <yixu310@gmail.com>

* Update src/diffusers/pipelines/llada2/pipeline_llada2.py

Co-authored-by: YiYi Xu <yixu310@gmail.com>

* Update src/diffusers/pipelines/llada2/pipeline_llada2.py

Co-authored-by: YiYi Xu <yixu310@gmail.com>

* Update src/diffusers/pipelines/llada2/pipeline_llada2.py

Co-authored-by: YiYi Xu <yixu310@gmail.com>

* Update src/diffusers/pipelines/llada2/pipeline_llada2.py

Co-authored-by: YiYi Xu <yixu310@gmail.com>

* Update src/diffusers/pipelines/llada2/pipeline_llada2.py

Co-authored-by: YiYi Xu <yixu310@gmail.com>

* Update src/diffusers/pipelines/llada2/pipeline_llada2.py

Co-authored-by: YiYi Xu <yixu310@gmail.com>

* Update src/diffusers/pipelines/llada2/pipeline_llada2.py

Co-authored-by: YiYi Xu <yixu310@gmail.com>

* Update src/diffusers/pipelines/llada2/pipeline_llada2.py

Co-authored-by: YiYi Xu <yixu310@gmail.com>

* Update src/diffusers/pipelines/llada2/pipeline_llada2.py

Co-authored-by: YiYi Xu <yixu310@gmail.com>

* Update src/diffusers/pipelines/llada2/pipeline_llada2.py

Co-authored-by: YiYi Xu <yixu310@gmail.com>

* Update src/diffusers/pipelines/llada2/pipeline_llada2.py

Co-authored-by: YiYi Xu <yixu310@gmail.com>

* Update src/diffusers/pipelines/llada2/pipeline_llada2.py

Co-authored-by: YiYi Xu <yixu310@gmail.com>

* Update src/diffusers/pipelines/llada2/pipeline_llada2.py

Co-authored-by: YiYi Xu <yixu310@gmail.com>

* Update src/diffusers/pipelines/llada2/pipeline_llada2.py

Co-authored-by: YiYi Xu <yixu310@gmail.com>

* removed DiscreteDiffusionPipelineMixin

* add support for 2d masks for flash attn

* Update src/diffusers/training_utils.py

Co-authored-by: dg845 <58458699+dg845@users.noreply.github.com>

* Update src/diffusers/training_utils.py

Co-authored-by: dg845 <58458699+dg845@users.noreply.github.com>

* fix issues from review

* added tests

* formatting

* add check_eos_finished to scheduler

* Update src/diffusers/pipelines/llada2/pipeline_llada2.py

Co-authored-by: dg845 <58458699+dg845@users.noreply.github.com>

* Update src/diffusers/pipelines/llada2/pipeline_llada2.py

Co-authored-by: dg845 <58458699+dg845@users.noreply.github.com>

* Update src/diffusers/pipelines/llada2/pipeline_llada2.py

Co-authored-by: dg845 <58458699+dg845@users.noreply.github.com>

* Update src/diffusers/pipelines/llada2/pipeline_llada2.py

Co-authored-by: dg845 <58458699+dg845@users.noreply.github.com>

* Update src/diffusers/schedulers/scheduling_block_refinement.py

Co-authored-by: dg845 <58458699+dg845@users.noreply.github.com>

* Update src/diffusers/pipelines/llada2/pipeline_llada2.py

Co-authored-by: dg845 <58458699+dg845@users.noreply.github.com>

* Update src/diffusers/pipelines/llada2/pipeline_llada2.py

Co-authored-by: dg845 <58458699+dg845@users.noreply.github.com>

* Update src/diffusers/pipelines/llada2/pipeline_llada2.py

Co-authored-by: dg845 <58458699+dg845@users.noreply.github.com>

* Update src/diffusers/schedulers/scheduling_block_refinement.py

Co-authored-by: dg845 <58458699+dg845@users.noreply.github.com>

* fix renaming issues and types

* remove duplicate check

* Update docs/source/en/api/pipelines/llada2.md

Co-authored-by: dg845 <58458699+dg845@users.noreply.github.com>

* Update src/diffusers/pipelines/llada2/pipeline_llada2.py

Co-authored-by: dg845 <58458699+dg845@users.noreply.github.com>

* Update src/diffusers/pipelines/llada2/pipeline_llada2.py

Co-authored-by: dg845 <58458699+dg845@users.noreply.github.com>

---------

Co-authored-by: YiYi Xu <yixu310@gmail.com>
Co-authored-by: dg845 <58458699+dg845@users.noreply.github.com>
 2026-03-25 16:17:50 +05:30
Sayak Paul
e358ddcce6 fix to device and to dtype tests. (#13323) 2026-03-25 11:47:02 +05:30
Sayak Paul
153fcbc5a8 fix klein lora loading. (#13313) 2026-03-25 07:51:35 +05:30
Beinsezii
da6718f080 ZImageTransformer2D: Only build attention mask if seqlens are not equal (#12955) 2026-03-24 06:06:50 -10:00
Alexey Kirillov
832676d35e Use defaultdict for _SET_ADAPTER_SCALE_FN_MAPPING (#13320) 
refactor: use defaultdict for _SET_ADAPTER_SCALE_FN_MAPPING

Co-authored-by: Alexkkir <alexkkir@gmail.coom>
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
 2026-03-24 17:49:50 +05:30
Dhruv Nair
7bbd96da5d [CI] Update fetching pipelines for latest HF Hub Version (#13322) 
update
 2026-03-24 16:42:32 +05:30
Dhruv Nair
62777fa819 Fix unguarded torchvision import in Cosmos (#13321) 
update
 2026-03-24 16:00:24 +05:30
Sayak Paul
f1fd515257 [tests] fix lora logging tests for models. (#13318) 
* fix lora logging tests for models.

* make style
 2026-03-24 15:48:03 +05:30
Cheung Ka Wai
afdda57f61 Fix the attention mask in ulysses SP for QwenImage (#13278) 
* fix mask in SP

* change the modification to qwen specific

* drop xfail since qwen-image mask is fixed

---------

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
 2026-03-24 02:12:50 -07:00
YangKai0616
5fc2bd2c8f Stabilize low-precision custom autoencoder RMS normalization (#13316) 
* Stabilize low-precision custom autoencoder RMS normalization

* Add fp8/4

* Apply style fixes

---------

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: dg845 <58458699+dg845@users.noreply.github.com>
 2026-03-24 02:00:05 -07:00
Sayak Paul
6350a7690a [chore] properly deprecate src.diffusers.utils.testing_utils. (#13314) 
properly deprecate src.diffusers.utils.testing_utils.
 2026-03-24 10:54:35 +05:30
Cheung Ka Wai
9d4c9dcf21 change QwenImageTransformer UT to batch inputs (#13312) 
* UT expands to batch inputs

* update according to suggestion

* update according to suggestion 2

* fix CI

* update according to suggestion 3

* clean line
 2026-03-24 08:56:40 +05:30
ddavidchick
ef309a1bb0 Add KVAE 1.0 (#13033) 
* add kvae2d

* add kvae3d video

* add docs for kvae2d and kvae3d video

* style fixes

* fix kvae3d docs

* fix normalzation

* fix kvae video for code style

* fix kvae video

* kvae minor fixes

* add gradient ckpting for kvaes

* get rid of inplace ops kvae video

* add tests for KVAEs

* kvae2d normalization style change

* kvaes fix style

* update dummy_pt_objects test for kvaes

---------

Co-authored-by: YiYi Xu <yixu310@gmail.com>
 2026-03-23 12:56:49 -10:00
Charles
b9761ce5a2 [export] Add export-safe LRU cache helper (#13290) 
* [core] Add export-safe LRU cache helper

* torch version check!

---------

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
 2026-03-23 18:10:07 +05:30
Dhruv Nair
52558b45d8 [CI] Flux2 Model Test Refactor (#13071) 
* update

* update

* update

---------

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
 2026-03-23 16:56:08 +05:30
Sayak Paul
c02c17c6ee [tests] test load_components in modular (#13245) 
* test load_components.

* fix

* fix

* u[

* up
 2026-03-21 09:41:48 +05:30
Sayak Paul
a9855c4204 [tests] fix audioldm2 tests. (#13293) 
fix audioldm2 tests.
 2026-03-20 20:53:21 +05:30
Sayak Paul
0b35834351 [core] fa4 support. (#13280) 
* start fa4 support.

* up

* specify minimum version
 2026-03-20 17:28:09 +05:30