mirror of https://github.com/vllm-project/vllm.git synced 2025-12-06 06:53:12 +08:00

Files

Harry Mellor ba5c5e5404 [Docs] Switch to better markdown linting pre-commit hook (#21851 )

Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>

2025-07-29 19:45:08 -07:00

Disaggregated Prefill V1

This example contains scripts that demonstrate disaggregated prefill in the offline setting of vLLM.

Files

run.sh - A helper script that will run prefill_example.py and decode_example.py sequentially.
- Make sure you are in the examples/offline_inference/disaggregated-prefill-v1 directory before running run.sh.
prefill_example.py - A script which performs prefill only, saving the KV state to the local_storage directory and the prompts to output.txt.
decode_example.py - A script which performs decode only, loading the KV state from the local_storage directory and the prompts from output.txt.