+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Inference Engines (https://www.datasculpting.ai/inference/forumdisplay.php?fid=9)
+--- Forum: Ollama / LM Studio / Desktop Runtimes (https://www.datasculpting.ai/inference/forumdisplay.php?fid=38)
+--- Thread: Ollama for power users on Threadripper box (/showthread.php?tid=66)
Ollama for power users on Threadripper box - embershed99 - 08-30-2026
This keeps coming up in conversation so I figured I'd start a thread.
One thing I keep noticing is that people underestimate long context until they actually try to run this stuff for more than one afternoon.
Wondering if this is just where the whole space is heading now.
This is mostly for local work rather than public serving.
Re: Ollama for power users on Threadripper box - amberhall - 08-30-2026
Tried to reproduce it just now. My result was mixed: first run looked bad, second run looked normal. The main thing that moved was latency, not quality.
Re: Ollama for power users on Threadripper box - edgekite - 08-30-2026
That matches what I was seeing. That matches what I saw on B200.
Re: Ollama for power users on Threadripper box - orbitbench6 - 08-30-2026
Appreciate the detail. I ended up in about the same place using SGLang. Worth rechecking your defaults though, because one bad assumption can make the rest of the thread look confusing.
Re: Ollama for power users on Threadripper box - edgeriver - 08-30-2026
Not sure I agree there. I've seen people blame batch size when the real issue was much more mundane. Could be a workload difference, but I wouldn't generalize too fast from one setup.
Re: Ollama for power users on Threadripper box - fusedhall - 08-31-2026
I tested this tonight. My result was mixed: first run looked bad, second run looked normal. The main thing that moved was latency, not quality.
Re: Ollama for power users on Threadripper box - vectorcrate98 - 08-31-2026
A couple things I'd check first. Before rebuilding anything, I'd test one clean baseline with Kimi and compare from there.
Re: Ollama for power users on Threadripper box - amberhall - 08-31-2026
Interesting, I hadn't tried that. The pattern around structured output has been pretty similar for me. I got better results once I stopped chasing one metric and just looked at whether the workflow felt usable.
Re: Ollama for power users on Threadripper box - edgeriver - 09-01-2026
Quote:Interesting, I hadn't tried that. The pattern around structured output has been pretty similar for me. I got better results once I stopped chasing one
worth checking the driver first.