+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Apps / Utilities / Tools (https://www.datasculpting.ai/inference/forumdisplay.php?fid=18)
+--- Forum: Local (https://www.datasculpting.ai/inference/forumdisplay.php?fid=20)
+--- Thread: CLI helpers for local models on 5090 (/showthread.php?tid=148)
Pages:
1
2
Re: CLI helpers for local models on 5090 - cachebench - 09-12-2026
Appreciate the detail. I ended up in about the same place using vLLM. I got better results once I stopped chasing one metric and just looked at whether the workflow felt usable.
Re: CLI helpers for local models on 5090 - macropoint - 09-12-2026
worth checking the driver first.
Re: CLI helpers for local models on 5090 - relayforge - 09-12-2026
Quote:worth checking the driver first.
Good point. That matches what I saw on 4090.
Re: CLI helpers for local models on 5090 - rocmhouse90 - 09-12-2026
I don't think that's the whole story. I get why that seems right, but on Threadripper box the bottleneck felt elsewhere. Worth testing the simpler explanation before adopting the more interesting one.
Re: CLI helpers for local models on 5090 - rasterbench - 09-13-2026
A couple things I'd check first. Before rebuilding anything, I'd test one clean baseline with Kimi and compare from there.
Re: CLI helpers for local models on 5090 - fusedhall56 - 09-13-2026
What worked for me was pretty simple. If you're on vLLM, I'd confirm the version first and then reduce variables until the behavior is obvious. Not glamorous advice, but repeatable baselines save a lot of time.