vllm-project/vllm
1
0
Fork 0
mirror of https://github.com/vllm-project/vllm.git synced 2025-12-06 06:53:12 +08:00
Code Issues Packages Projects Releases Wiki Activity
11,993 Commits 170 Branches 98 Tags
main
Commit Graph

515 Commits

Author SHA1 Message Date
Varun Sundar Rabindranath
19bee6d12d [Performance][DP/EP] Add silu_mul_per_token_group_quant_fp8_colmajor kernel (#29470) 
Signed-off-by: Varun Sundar Rabindranath <vsundarr@redhat.com>
Co-authored-by: Varun Sundar Rabindranath <vsundarr@redhat.com>
Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com>
 2025-12-03 18:04:59 +00:00
Lumis Chen
9bcf92295a [Core] Add xxHash as a high-performance hash option for accelerating prefix caching (#29163) 
Signed-off-by: LuminolT <lumischen01@gmail.com>
Signed-off-by: Lumis Chen <lumischen01@gmail.com>
Co-authored-by: Russell Bryant <rbryant@redhat.com>
 2025-12-03 16:06:57 +00:00
Harry Mellor
951445a52d Remove default values from InitVars so that they're not stored (#29859) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-12-02 12:16:37 +00:00
Cyrus Leung
653591d5e7 [Chore] Move tokenizer initialization methods (#29793) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-12-02 13:33:37 +08:00
Liu Jinyi
f5516039c5 [Doc] fix heading levels (#29783) 
Signed-off-by: KKKZOZ <kkkzoz@qq.com>
 2025-12-01 14:49:22 +00:00
Jinzhen Lin
1656ad3704 [Kernel][Quantization] add w4a8 support for marlin kernel (#24722) 
Signed-off-by: Jinzhen Lin <jinzhen.ljz@antgroup.com>
Signed-off-by: Michael Goin <mgoin64@gmail.com>
Signed-off-by: Jinzhen Lin <linjinzhen@hotmail.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
Co-authored-by: Michael Goin <mgoin@redhat.com>
 2025-11-29 07:19:33 -08:00
Cyrus Leung
34a984274e [Misc] Refactor tokenizer interface (#29693) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-11-29 04:02:21 -08:00
Didier Durand
eca7a8fb59 [Doc]: fix typos in various files (#29230) 
Signed-off-by: Didier Durand <durand.didier@gmail.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-11-24 11:10:48 +00:00
Pan Li
e5bfcb6a88 [BugFix][PD]: make example proxy usable with P2pNcclConnector (#26628) 
Signed-off-by: PAN <1162953505@qq.com>
 2025-11-20 17:38:31 +00:00
Harry Mellor
a8b70304d6 Update rope_scaling to rope_parameters in preparation for Transformers v5 (#28542) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-11-19 09:06:36 -08:00
Kunshang Ji
2a2d5d2780 Replace torch.cuda.Event with torch.Event for better hardware compatibility (#26985) 
Signed-off-by: Kunshang Ji <kunshang.ji@intel.com>
 2025-11-18 11:34:36 -08:00
Ido Segev
49a986ecd4 [Benchmark] multi_turn: Report warmup-inclusive runtime (#28937) 
Signed-off-by: Ido Segev <idos@pliops.com>
 2025-11-18 16:38:22 +00:00
ai-jz
d231876ce3 [Benchmark] Fix client seed synchronization in multi-turn benchmark (#28512) 
Signed-off-by: ai-jz <aijz.xplr@gmail.com>
 2025-11-16 15:04:32 +08:00
Bram Wasti
f849ee739c Adding a benchmark for batch invariance (#28161) 
Signed-off-by: Bram Wasti <bwasti@meta.com>
Signed-off-by: Bram Wasti <bwasti@fb.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2025-11-16 13:22:17 +08:00
Di Wu
e63fd44560 Fix: Correctly filter special tokens in benchmark_prefix_caching (#28615) 
Signed-off-by: Di Wu <dw2761@nyu.edu>
 2025-11-13 10:57:44 +00:00
ai-jz
f31419ed8b [Benchmark] Add retry support to fix workload bias in multi-turn benchmark (#28493) 2025-11-12 05:00:45 +00:00
Michael Goin
f9a4087182 Remove weight_scale.T special case for SM90 Block FP8 CUTLASS kernel (#28431) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2025-11-11 11:46:04 -05:00
Ido Segev
3380543b20 Add request timeout override for multi-turn benchmarks (#28386) 
Signed-off-by: Ido Segev <idos@pliops.com>
 2025-11-11 13:41:18 +00:00
Ido Segev
d381eb967f Multi turn benchmark progress bar for synthetic conversation generation (#28394) 
Signed-off-by: Ido Segev <idos@pliops.com>
 2025-11-11 11:06:04 +00:00
Xin Yang
57201a6a4c Fix rotary embedding benchmark script (#28323) 
Signed-off-by: Xin Yang <xyangx@amazon.com>
 2025-11-10 21:57:12 -05:00
Ilya Markov
d17ecc6b19 [PERF] Allreduce fusion. Support torch native matching. Tuning of the thresholds (#24248) 
Signed-off-by: Luka Govedič <lgovedic@redhat.com>
Signed-off-by: Luka Govedič <ProExpertProg@users.noreply.github.com>
Signed-off-by: ilmarkov <markovilya197@gmail.com>
Co-authored-by: Luka Govedič <lgovedic@redhat.com>
Co-authored-by: Luka Govedič <ProExpertProg@users.noreply.github.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
 2025-11-10 18:33:11 -05:00
Xiake Sun
03fa4d3fb3 [Hardware][AMD][Model] Add Triton MoE tuning support and optimized configs for Qwen3 omni for MI308X (#28373) 
Signed-off-by: Xiake Sun <xiake.sun@amd.com>
Signed-off-by: Xiake Sun <xisun@amd.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2025-11-10 04:53:40 +00:00
Michael Goin
f32229293e Disable nm-testing models with issues in CI (#28206) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2025-11-06 06:19:07 -08:00
tomeras91
e4ee658672 [Model] add optimal triton fused moe configs for NemotronH MoE (#27967) 
Signed-off-by: Tomer Asida <57313761+tomeras91@users.noreply.github.com>
 2025-11-04 12:59:43 +00:00
yugong333
2ec401bc39 Load tuned fused_moe_lora shrink and expand kernel configs separately (#27435) 
Signed-off-by: Yu Gong <yu3.gong@gmail.com>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-11-04 18:27:35 +08:00
ai-jz
2c0c7c39bd feat(benchmarks): support HF model names in multi-turn benchmark (#27850) 2025-11-01 08:04:52 +00:00
Yeshwanth N
71b1c8b667 [Chore]:Extract math and argparse utilities to separate modules (#27188) 
Signed-off-by: Yeshwanth Surya <yeshsurya@gmail.com>
Signed-off-by: Yeshwanth N <yeshsurya@gmail.com>
Signed-off-by: yeshsurya <yeshsurya@gmail.com>
 2025-10-26 04:03:32 -07:00
Isotr0py
6ac5e06f7c [Chore] Clean up pytorch helper functions in vllm.utils (#26908) 
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Signed-off-by: isotr0py <2037008807@qq.com>
 2025-10-18 09:48:22 -07:00
Isotr0py
3125d79950 [Chore] Remove unused PolyNorm layer (#27110) 
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
 2025-10-17 19:03:43 +00:00
Harry Mellor
6c9fdbf725 [Docs] Replace rst style double-backtick with md single-backtick (#27091) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-10-17 02:47:34 -07:00
Lukas Geiger
ed344f4116 Cleanup code after Python 3.10 upgrade (#26520) 
Signed-off-by: Lukas Geiger <lukas.geiger94@gmail.com>
 2025-10-16 03:38:23 -07:00
wangxiyuan
8f4b313c37 [Misc] rename torch_dtype to dtype (#26695) 
Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>
 2025-10-15 12:11:48 +00:00
Harry Mellor
8fcaaf6a16 Update Optional[x] -> x | None and Union[x, y] to x | y (#26633) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-10-12 09:51:31 -07:00
Roberto L. Castro
96ad65b7fe [Transform] [Quantization] Add QuTLASS support to vLLM (#24440) 
Signed-off-by: LopezCastroRoberto <roberto.lopez.castro@udc.es>
Signed-off-by: Roberto L. Castro <38211239+LopezCastroRoberto@users.noreply.github.com>
Signed-off-by: Andrei Panferov <andrei@panferov.org>
Co-authored-by: Andrei Panferov <andrei@panferov.org>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
 2025-10-10 09:43:40 -07:00
Elvir Crnčević
7b03584de8 Silu v2 (#25074) 
Signed-off-by: mgoin <mgoin64@gmail.com>
Signed-off-by: elvircrn <elvircrn@gmail.com>
Signed-off-by: Elvir Crnčević <elvircrn@gmail.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Varun Sundar Rabindranath <varunsundar08@gmail.com>
 2025-10-10 15:19:53 +00:00
Harry Mellor
e09d1753ec Remove Python 3.9 support ahead of PyTorch 2.9 in v0.11.1 (#26416) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-10-08 10:40:42 -07:00
Lukas Geiger
6273fe8d3d [Benchmarks] Fix imports in FP8 tuning script (#26407) 
Signed-off-by: Lukas Geiger <lukas.geiger94@gmail.com>
 2025-10-08 16:31:59 +00:00
Lukas Geiger
338b1bf04f [Benchmarks] Add support for Qwen 3 VL MoE tuning (#26419) 
Signed-off-by: Lukas Geiger <lukas.geiger94@gmail.com>
 2025-10-08 14:01:08 +00:00
Cyrus Leung
7e4cd070b0 [V0 Deprecation] Remove VLLM_USE_V1 from docs and scripts (#26336) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-10-07 16:46:44 +08:00
Karan Goel
824a3f403f [Misc] auto_tune: kill specific vllm process (#26304) 
Signed-off-by: Karan Goel <karangoel@google.com>
 2025-10-06 18:02:51 +00:00
Harry Mellor
6c04638214 Fix per file ruff ignores related to line length (#26262) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-10-06 05:12:40 +00:00
Harry Mellor
557b2e961d Remove all cases of fmt: on/off (#26253) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-10-05 09:18:14 -07:00
Harry Mellor
d6953beb91 Convert formatting to use ruff instead of yapf + isort (#26247) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-10-05 07:06:22 -07:00
Jiangyun Zhu
eb0fa43868 [Perf] Optimize reshape_and_cache CUDA Kernel (#25955) 
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
Co-authored-by: Liu-congo <1502632128@qq.com>
 2025-10-03 01:33:46 -07:00
ElizaWszola
502640c3f9 [Perf] Fix and reapply move apply w8a8 block fp8 linear to class (#25696) 
Signed-off-by: ElizaWszola <ewszola@redhat.com>
Signed-off-by: ElizaWszola <elizaw.9289@gmail.com>
Signed-off-by: Luka Govedič <lgovedic@redhat.com>
Signed-off-by: Luka Govedič <ProExpertProg@users.noreply.github.com>
Co-authored-by: Luka Govedič <ProExpertProg@users.noreply.github.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
Co-authored-by: Luka Govedič <lgovedic@redhat.com>
 2025-10-02 19:35:13 +00:00
Cyrus Leung
d00d652998 [CI/Build] Replace vllm.entrypoints.openai.api_server entrypoint with vllm serve command (#25967) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-10-02 10:04:57 -07:00
Jee Jee Li
67f3fb0844 [Bench] Add DeepSeekV32 to MoE benchmark (#25962) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-09-30 14:13:48 -07:00
Ekagra Ranjan
e71b8e210d [Spec Decode] Add Batch Parallel Ngram. Upto 8x lower overhead. (#24986) 
Signed-off-by: Ekagra Ranjan <3116519+ekagra-ranjan@users.noreply.github.com>
Co-authored-by: Nick Hill <nhill@redhat.com>
 2025-09-25 15:22:03 -07:00
Cyrus Leung
2f17117606 [mypy] Fix wrong type annotations related to tuple (#25660) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-09-25 13:00:45 +00:00
Tyler Michael Smith
1260180c67 Revert "[Performance] Move apply_w8a8_block_fp8_linear to an op class… (#25607) 
Signed-off-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>
 2025-09-25 08:05:21 +00:00