release notes that matter

[southden]

Ran a quick check on my box. My result was mixed: first run looked bad, second run looked normal. The surprise was that memory pressure showed up before throughput collapsed.

[routecraft]

Ran a quick check on my box. My result was mixed: first run looked bad, second run looked normal. The surprise was that memory pressure showed up befo

Thanks, that helps. That matches what I saw on A6000. Worth rechecking your defaults though, because one bad assumption can make the rest of the thread look confusing.

[routecraft]

[quote]Ran a quick check on my box. My result was mixed: first run looked bad, second run looked normal. The surprise was that memory pressure showed

Thanks, that helps. I ended up in about the same place using vLLM. Worth rechecking your defaults though, because one bad assumption can make the rest of the thread look confusing.

[oxidestream]

that fixed it for me too.

[routecraft]

A couple things I'd check first. Start by verifying the driver/runtime combo, then test with a smaller prompt, then look at router policy. A lot of these threads end up being one bad config plus a pile of conclusions drawn from it. Turns out 'just one more test' was a lie.

[routecraft]

What exact setup are you running? What model, what runtime, and roughly what context size?