+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Inference Engines (https://www.datasculpting.ai/inference/forumdisplay.php?fid=9)
+--- Forum: Other Engines (https://www.datasculpting.ai/inference/forumdisplay.php?fid=39)
+--- Thread: prefill bottlenecks on longer prompts with vLLM (/showthread.php?tid=9)
Pages:
1
2
Re: prefill bottlenecks on longer prompts with vLLM - batchshed - 08-26-2026
same here on my side.
Re: prefill bottlenecks on longer prompts with vLLM - orbitgarden - 08-27-2026
worth checking the driver first.
Re: prefill bottlenecks on longer prompts with vLLM - routerunner - 08-29-2026
Tried to reproduce it just now. On 7900 XTX with PyTorch, I saw basically the same behavior. The main thing that moved was latency, not quality.