|
tokens/sec tuning ideas: what are you seeing?
|
10-18-2026, 07:28 PM
Quote:What worked for me was pretty simple. On B200, the easiest win is often to back off the aggressive settings and see whether the problem survives. Not Tried to reproduce it just now. I couldn't reproduce the slowdown until I pushed context much higher. I expected the model choice to matter more than it did. |
|
« Next Oldest | Next Newest »
|
| Messages In This Thread |
|
tokens/sec tuning ideas: what are you seeing? - by circuitkite - 10-18-2026, 05:28 PM
Re: tokens/sec tuning ideas: what are you seeing? - by stackpilot - 10-18-2026, 06:49 PM
Re: tokens/sec tuning ideas: what are you seeing? - by audiopoint18 - 10-18-2026, 07:28 PM
Re: tokens/sec tuning ideas: what are you seeing? - by turbosupply83 - 10-18-2026, 08:02 PM
Re: tokens/sec tuning ideas: what are you seeing? - by vramport - 10-19-2026, 12:34 AM
Re: tokens/sec tuning ideas: what are you seeing? - by vrambandit - 10-19-2026, 01:24 AM
Re: tokens/sec tuning ideas: what are you seeing? - by torchstation - 10-19-2026, 03:32 AM
Re: tokens/sec tuning ideas: what are you seeing? - by turbosupply83 - 10-19-2026, 04:13 AM
Re: tokens/sec tuning ideas: what are you seeing? - by tinycontext - 10-18-2026, 08:16 PM
|
