Discussions

• 1 You must be logged in to vote

  🙏

  Plugins and addons

  carelesswhisp announced Mar 20, 2025 in Q&A

  

  

  

  

  2
• 3 You must be logged in to vote

  🙏

  Reasoning models and structured output (e.g. QwQ-32B and JSON)

  matthew-at-qamcom announced Mar 27, 2025 in Q&A · Closed

  

  

  4
• 11 You must be logged in to vote

  🙏

  How TOKENIZERS_PARALLELISM will affect vLLM?

  Junjie-Chu announced Mar 20, 2024 in Q&A

  

  

  1
• 3 You must be logged in to vote

  💬

  0.7.4 开发版CPU使用率一直100%，即使没处理请求的时候也一样

  AndrewTsao announced Mar 14, 2025 in General · Closed

  

  

  1
• 1 You must be logged in to vote

  🙏

  is there a way to run vllm without torch.compiled model?

  carlesoctav announced Dec 10, 2024 in Q&A

  

  

  1
• 1 You must be logged in to vote

  🙏

  Running Llama4 quantized on 2xH100 80GB

  ilyabcodin announced May 4, 2025 in Q&A · Closed

  

  

  0
• 1 You must be logged in to vote

  🙏

  WARNING 02-09 10:24:45 cpu.py:99] ray is not supported on CPU, fallback to mp distributed executor backend.

  jdmedeiros announced Feb 9, 2025 in Q&A

  

  

  2
• 3 You must be logged in to vote

  🙏

  v0.8.2 "No available memory for the cache blocks." when launching models on the same GPU

  CaoChensy announced Apr 1, 2025 in Q&A

  

  

  

  

  3
• 1 You must be logged in to vote

  💬

  Why gfx908 (mi100) was deleted from the arch list?

  hnhyzz announced Apr 28, 2025 in General

  

  

  0
• 2 You must be logged in to vote

  🙏

  Any known integration with n8n ?

  wugutech announced Apr 28, 2025 in Q&A

  

  

  0
• 1 You must be logged in to vote

  💬

  I just published a performance test result of vllm vs sglang but can someone help me explain it?

  qiulang announced Apr 26, 2025 in General · Closed

  

  

  1
• 1 You must be logged in to vote

  🙏

  Many 0 Day user questions - What is this vllm thing useful

  ket395 announced Apr 26, 2025 in Q&A

  

  

  0
• 4 You must be logged in to vote

  🙏

  Does vllm support CPU?

  gokul427 announced Sep 9, 2023 in Q&A · Closed

  

  

  

  

  7
• 2 You must be logged in to vote

  🙏

  LoRA Adapter enabling with vLLM is not working

  SSPRATAP17 announced Apr 14, 2025 in Q&A

  

  

  

  

  3
• 4 You must be logged in to vote

  🙏

  KV cache in VLLM serve

  farouk09 announced Apr 23, 2025 in Q&A

  

  

  0
• 2 You must be logged in to vote

  🙏

  VLLM Inference Optimizations

  wlm64 announced Apr 21, 2025 in Q&A

  

  

  0
• 1 You must be logged in to vote

  🙏

  Doing Batched Inference Using TokensPrompt

  wlm64 announced Apr 10, 2025 in Q&A

  

  

  1
• 1 You must be logged in to vote

  🙏

  Does VLLM support DP attention for deepseek?

  llc-kc announced Apr 18, 2025 in Q&A

  

  

  0
• 1 You must be logged in to vote

  🙏

  How to configure vllm to gracefully mark the too-long inputs without throwing?

  vadimkantorov announced Apr 16, 2025 in Q&A

  

  

  0
• 1 You must be logged in to vote

  🙏

  Why my PR build docker image failed, how to solve this problem?

  xsank announced Apr 15, 2025 in Q&A · Closed

  

  

  0
• 1 You must be logged in to vote

  🙏

  The num of dynamically serving LoRA adapters can not limited by param max-loras ?

  estuday announced Apr 15, 2025 in Q&A

  

  

  0
• 1 You must be logged in to vote

  💬

  I use VLLM to deploy DeepSeek, and when there are high concurrency requests, many of them are pending reqs. How can I set a timeout for these waiting requests?

  nvliajia announced Apr 10, 2025 in General

  

  

  0
• 1 You must be logged in to vote

  🙏

  Running Custom Model with isinstance(model, VllmModelForTextGeneration) problem

  yesilcagri announced Apr 1, 2025 in Q&A

  

  

  

  

  2
• 1 You must be logged in to vote

  💬

  q_lora_rank in vLLM with DeepSeek MLA Model

  erictanjn announced Apr 7, 2025 in General

  

  

  0
• 2 You must be logged in to vote

  🙏

  Docker Image Build

  artist-genai announced Feb 28, 2025 in Q&A

  

  

  2