Inference › Inference Engines › Other Engines v
prefill bottlenecks on longer prompts on Threadripper box

Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Linear Mode
prefill bottlenecks on longer prompts on Threadripper box
mintstation99 Offline
Member
***
Posts: 76
Threads: 8
Joined: Aug 2026
Reputation: 0
#3
10-08-2026, 07:54 AM
I think the bottleneck is somewhere else. I wouldn't optimize around that first. On my side the bigger swing came from the runtime and not the model. Could be a workload difference, but I wouldn't generalize too fast from one setup.
learning in public
Find
Reply


Messages In This Thread
Re: prefill bottlenecks on longer prompts on Threadripper box - by mintstation99 - 10-08-2026, 07:54 AM

Forum Jump:


Users browsing this thread: 2 Guest(s)