Commit 789f1e8
Minsung-commit
Fix block_size initialization in KVCacheManager
Use actual KV cache block size from kv_cache_config instead of hash_block_size.
**Issue**: The previous implementation incorrectly used `hash_block_size` for
token metrics calculation. The hash_block_size is used for hashing granularity,
not for the actual KV cache block size used by BlockPool.
**Fix**: Initialize `self.block_size` from `kv_cache_config.kv_cache_groups[].kv_cache_spec.block_size`,
which represents the actual block size used for token storage.
**Impact**: This ensures token-level metrics (total_tokens, used_tokens, free_tokens)
accurately reflect the real KV cache capacity, especially for models using larger
block sizes than the hash granularity.
Addresses bot review feedback on PR vllm-project#29836.
Signed-off-by: Minsung-commit <[email protected]>1 parent 3800caf commit 789f1e8
1 file changed
+14
-1
lines changed| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
106 | 106 | | |
107 | 107 | | |
108 | 108 | | |
109 | | - | |
110 | 109 | | |
111 | 110 | | |
| 111 | + | |
| 112 | + | |
| 113 | + | |
| 114 | + | |
| 115 | + | |
| 116 | + | |
| 117 | + | |
| 118 | + | |
| 119 | + | |
| 120 | + | |
| 121 | + | |
| 122 | + | |
| 123 | + | |
| 124 | + | |
112 | 125 | | |
113 | 126 | | |
114 | 127 | | |
| |||
0 commit comments