Inference › Inference Engines › Other Engines v
tokens/sec tuning ideas

Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Linear Mode
tokens/sec tuning ideas
northquill Offline
Junior Member
**
Posts: 35
Threads: 3
Joined: Sep 2026
Reputation: 0
#4
10-18-2026, 09:46 PM
I tested this tonight. On Threadripper box with SGLang, I saw basically the same behavior. The surprise was that memory pressure showed up before throughput collapsed. At this point the real bottleneck is my patience.
Find
Reply


Messages In This Thread
tokens/sec tuning ideas - by kernelmike - 10-18-2026, 04:52 PM
Re: tokens/sec tuning ideas - by northgarden37 - 10-18-2026, 06:00 PM
Re: tokens/sec tuning ideas - by sysstation69 - 10-18-2026, 06:05 PM
Re: tokens/sec tuning ideas - by northquill - 10-18-2026, 09:46 PM
Re: tokens/sec tuning ideas - by cacheworks - 10-19-2026, 12:00 AM
Re: tokens/sec tuning ideas - by rocmplane - 10-19-2026, 01:55 AM
Re: tokens/sec tuning ideas - by cachebench - 10-19-2026, 02:16 AM
Re: tokens/sec tuning ideas - by amberpatch25 - 10-19-2026, 03:23 AM
Re: tokens/sec tuning ideas - by vramstream - 10-19-2026, 04:24 AM

Forum Jump:


Users browsing this thread: 1 Guest(s)