+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Evaluation & Testing (https://www.datasculpting.ai/inference/forumdisplay.php?fid=16)
+--- Forum: Benchmarks (https://www.datasculpting.ai/inference/forumdisplay.php?fid=65)
+--- Thread: comparing models without fooling yourself on Threadripper box (/showthread.php?tid=647)
Pages:
1
2
comparing models without fooling yourself on Threadripper box - kernelghost - 10-02-2026
Spent some time on this today and wanted to compare notes.
Two paths both seem viable, but they break down differently once prefill/decode split becomes important.
Interested in tradeoffs more than fan loyalty.
My use case is pretty ordinary: local chat, some document work, and occasional batch jobs.
Re: comparing models without fooling yourself on Threadripper box - sparrowworks88 - 10-02-2026
Appreciate the detail. I don't think your conclusion is far off.
Re: comparing models without fooling yourself on Threadripper box - quietcraft - 10-02-2026
A couple things I'd check first. On Threadripper box, the easiest win is often to back off the aggressive settings and see whether the problem survives. A lot of these threads end up being one bad config plus a pile of conclusions drawn from it.
Re: comparing models without fooling yourself on Threadripper box - griffinloom66 - 10-02-2026
Quote:A couple things I'd check first. On Threadripper box, the easiest win is often to back off the aggressive settings and see whether the problem survive
Interesting, I hadn't tried that. The pattern around INT4 has been pretty similar for me. The biggest change for me was simplifying the stack instead of adding one more optimization layer.
Re: comparing models without fooling yourself on Threadripper box - labline - 10-02-2026
Quote:[quote]A couple things I'd check first. On Threadripper box, the easiest win is often to back off the aggressive settings and see whether the problem
I'd approach it in a boring order. On 7900 XTX, the easiest win is often to back off the aggressive settings and see whether the problem survives. Not glamorous advice, but repeatable baselines save a lot of time.
Re: comparing models without fooling yourself on Threadripper box - cudascope - 10-05-2026
My first guess would be this: On 7900 XTX, the easiest win is often to back off the aggressive settings and see whether the problem survives. Not glamorous advice, but repeatable baselines save a lot of time.
Re: comparing models without fooling yourself on Threadripper box - cacheline - 10-06-2026
Quote:My first guess would be this: On 7900 XTX, the easiest win is often to back off the aggressive settings and see whether the problem survives. Not glam
subscribed, curious where this lands.
Re: comparing models without fooling yourself on Threadripper box - batchfield55 - 10-06-2026
subscribed, curious where this lands.
There are a few different conversations hiding inside this thread, so being explicit about the goal helps.
Re: comparing models without fooling yourself on Threadripper box - northgrid33 - 10-06-2026
Quote:subscribed, curious where this lands.
Did a small comparison this afternoon. I couldn't reproduce the slowdown until I pushed context much higher.
Re: comparing models without fooling yourself on Threadripper box - deltastation - 10-06-2026
Not sure I agree there. I've seen people blame structured output when the real issue was much more mundane.