Inference › Inference Engines › Other Engines v
prefill bottlenecks on longer prompts with vLLM

Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Threaded Mode
prefill bottlenecks on longer prompts with vLLM
batchshed Offline
Member
***
Posts: 74
Threads: 9
Joined: Sep 2026
Reputation: 0
#11
08-26-2026, 06:28 PM
same here on my side.
learning in public
Find
Reply
orbitgarden Offline
Junior Member
**
Posts: 41
Threads: 7
Joined: Sep 2026
Reputation: 0
#12
08-27-2026, 07:00 PM
worth checking the driver first.
curious and caffeinated
Find
Reply
routerunner Offline
Junior Member
**
Posts: 15
Threads: 2
Joined: Sep 2026
Reputation: 0
#13
08-29-2026, 07:07 AM
Tried to reproduce it just now. On 7900 XTX with PyTorch, I saw basically the same behavior. The main thing that moved was latency, not quality.
still benchmarking everything
Find
Reply


Forum Jump:


Users browsing this thread: 1 Guest(s)