Inference › Inference Engines › Other Engines v
prefill bottlenecks on longer prompts on Threadripper box

Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Linear Mode
prefill bottlenecks on longer prompts on Threadripper box
vramarch Offline
Junior Member
**
Posts: 49
Threads: 6
Joined: Sep 2026
Reputation: 0
#1
10-06-2026, 06:56 PM
Spent some time on this today and wanted to compare notes.

Two paths both seem viable, but they break down differently once VRAM headroom becomes important.

What would you pick if you had to live with it for a few months?
Find
Reply


Messages In This Thread
prefill bottlenecks on longer prompts on Threadripper box - by vramarch - 10-06-2026, 06:56 PM

Forum Jump:


Users browsing this thread: 1 Guest(s)