Inference › Inference Engines › Other Engines v
tokens/sec tuning ideas

Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Linear Mode
tokens/sec tuning ideas
cacheline Offline
Senior Member
****
Posts: 341
Threads: 51
Joined: Sep 2026
Reputation: 0
#12
10-06-2026, 06:25 PM
Quote:subscribed, curious where this lands.

Tried to reproduce it just now. My result was mixed: first run looked bad, second run looked normal. The surprise was that memory pressure showed up before throughput collapsed.
measure twice, benchmark three times
Find
Reply


Messages In This Thread
tokens/sec tuning ideas - by latentgarden - 10-01-2026, 01:09 AM
Re: tokens/sec tuning ideas - by dockkite69 - 10-01-2026, 05:13 AM
Re: tokens/sec tuning ideas - by cobalttinker - 10-01-2026, 05:21 AM
Re: tokens/sec tuning ideas - by quiettoken - 10-02-2026, 06:00 PM
Re: tokens/sec tuning ideas - by visionforge - 10-02-2026, 07:01 PM
Re: tokens/sec tuning ideas - by fusedriver - 10-03-2026, 07:10 AM
Re: tokens/sec tuning ideas - by visionforge - 10-03-2026, 07:36 AM
Re: tokens/sec tuning ideas - by labline - 10-03-2026, 03:49 PM
Re: tokens/sec tuning ideas - by fusedriver - 10-04-2026, 04:12 PM
Re: tokens/sec tuning ideas - by llm_wren - 10-06-2026, 05:22 PM
Re: tokens/sec tuning ideas - by oxideworks59 - 10-06-2026, 04:46 AM
Re: tokens/sec tuning ideas - by cacheline - 10-06-2026, 06:25 PM
Re: tokens/sec tuning ideas - by deltastation - 10-07-2026, 06:38 AM
Re: tokens/sec tuning ideas - by mintforge - 10-10-2026, 06:56 AM
Re: tokens/sec tuning ideas - by lablab - 10-10-2026, 07:07 PM
Re: tokens/sec tuning ideas - by vectorpoint - 10-10-2026, 11:31 PM

Forum Jump:


Users browsing this thread: 1 Guest(s)