|
tokens/sec tuning ideas: what are you seeing?
|
10-19-2026, 03:32 AM
Quote:[quote][quote]What worked for me was pretty simple. On B200, the easiest win is often to back off the aggressive settings and see whether the problem I tested this tonight. On mini PC with SGLang, I saw basically the same behavior. The surprise was that memory pressure showed up before throughput collapsed. |
|
« Next Oldest | Next Newest »
|
| Messages In This Thread |
|
tokens/sec tuning ideas: what are you seeing? - by circuitkite - 10-18-2026, 05:28 PM
Re: tokens/sec tuning ideas: what are you seeing? - by stackpilot - 10-18-2026, 06:49 PM
Re: tokens/sec tuning ideas: what are you seeing? - by audiopoint18 - 10-18-2026, 07:28 PM
Re: tokens/sec tuning ideas: what are you seeing? - by turbosupply83 - 10-18-2026, 08:02 PM
Re: tokens/sec tuning ideas: what are you seeing? - by vramport - 10-19-2026, 12:34 AM
Re: tokens/sec tuning ideas: what are you seeing? - by vrambandit - 10-19-2026, 01:24 AM
Re: tokens/sec tuning ideas: what are you seeing? - by torchstation - 10-19-2026, 03:32 AM
Re: tokens/sec tuning ideas: what are you seeing? - by turbosupply83 - 10-19-2026, 04:13 AM
Re: tokens/sec tuning ideas: what are you seeing? - by tinycontext - 10-18-2026, 08:16 PM
|
