latest release actually worth attention on Threadripper box

[vrambandit]

Did a bit of testing and thought I'd post the result here.

After a few runs on Threadripper box, I ended up with decent performance, but the setup only really made sense once I adjusted around NVFP4.

Curious if anyone else landed in the same place.

My use case is pretty ordinary: local chat, some document work, and occasional batch jobs.

[audiopoint18]

Interesting, I hadn't tried that. That matches what I saw on dual 3090. Worth rechecking your defaults though, because one bad assumption can make the rest of the thread look confusing.

[deepseeked]

What worked for me was pretty simple. On 7900 XTX, the easiest win is often to back off the aggressive settings and see whether the problem survives. The logs were somehow calmer than I was.

[mangotinker73]

Appreciate the detail. That matches what I saw on 5090. Worth rechecking your defaults though, because one bad assumption can make the rest of the thread look confusing.

[deepseeked]

that fixed it for me too.

[fusedriver]

I'd start with the simpler stack.

[amberrunner]

Tried to reproduce it just now. The issue went away once I changed one setting and restarted the stack cleanly. The main thing that moved was latency, not quality.

[phaseshed]

Can you fill in a bit more context? Do you see the same thing with a smaller batch or shorter prompt?

[deepseeked]

Interesting, I hadn't tried that. I don't think your conclusion is far off.

[routearch]

My first guess would be this: If you're on SGLang, I'd confirm the version first and then reduce variables until the behavior is obvious. Not glamorous advice, but repeatable baselines save a lot of time.