|
latest release actually worth attention
|
10-05-2026, 03:33 AM
Spent some time on this today and wanted to compare notes.
Two paths both seem viable, but they break down differently once INT4 becomes important. What would you pick if you had to live with it for a few months?
10-05-2026, 04:52 AM
What worked for me was pretty simple. If you're on SGLang, I'd confirm the version first and then reduce variables until the behavior is obvious. Not glamorous advice, but repeatable baselines save a lot of time.
10-05-2026, 05:47 AM
I'd approach it in a boring order. On 5090, the easiest win is often to back off the aggressive settings and see whether the problem survives.
watching release notes so you don't have to
10-05-2026, 07:16 AM
What worked for me was pretty simple. Before rebuilding anything, I'd test one clean baseline with FLUX and compare from there. A lot of these threads end up being one bad config plus a pile of conclusions drawn from it.
10-05-2026, 07:46 AM
Can you fill in a bit more context? Are you testing this on GB200 or something else?
watching release notes so you don't have to
10-05-2026, 04:11 PM
Quote:Can you fill in a bit more context? Are you testing this on GB200 or something else? What exact setup are you running? Do you see the same thing with a smaller batch or shorter prompt?
10-05-2026, 08:29 PM
following this because I'm in the same boat.
watching release notes so you don't have to
10-07-2026, 08:54 AM
I think the bottleneck is somewhere else. I wouldn't optimize around that first. On my side the bigger swing came from the runtime and not the model.
learning in public
10-08-2026, 09:43 PM
I tested this tonight. I couldn't reproduce the slowdown until I pushed context much higher.
trying not to rebuild again
|
|
« Next Oldest | Next Newest »
|
