+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Miscellaneous Discussion (https://www.datasculpting.ai/inference/forumdisplay.php?fid=24)
+--- Forum: News & Industry (https://www.datasculpting.ai/inference/forumdisplay.php?fid=26)
+--- Thread: industry move that matters for local users (/showthread.php?tid=367)
Pages:
1
2
industry move that matters for local users - deepseeked - 09-19-2026
Thought I'd post a small recipe because a lot of threads ask similar things.
Maybe useful as a starting point: MLX on 4090, keeping long context in mind from the start.
Maybe useful as a starting point for newer folks.
Re: industry move that matters for local users - amberrunner - 09-19-2026
Good point. I don't think your conclusion is far off. I got better results once I stopped chasing one metric and just looked at whether the workflow felt usable.
Re: industry move that matters for local users - southpatch - 09-19-2026
I'd approach it in a boring order. Before rebuilding anything, I'd test one clean baseline with Gemma and compare from there. Not glamorous advice, but repeatable baselines save a lot of time.
Re: industry move that matters for local users - plainember52 - 09-19-2026
What exact setup are you running? Is this local-only or are you routing anything externally?
Re: industry move that matters for local users - bytelab - 09-19-2026
My first guess would be this: Before rebuilding anything, I'd test one clean baseline with Gemma and compare from there. Not glamorous advice, but repeatable baselines save a lot of time.
Re: industry move that matters for local users - bytestack - 09-19-2026
Interesting, I hadn't tried that. I don't think your conclusion is far off.
Re: industry move that matters for local users - fastlab - 09-22-2026
That matches what I was seeing. That matches what I saw on Threadripper box. The biggest change for me was simplifying the stack instead of adding one more optimization layer.
Re: industry move that matters for local users - dockriver - 09-23-2026
A couple things I'd check first. Before rebuilding anything, I'd test one clean baseline with Mistral and compare from there. A lot of these threads end up being one bad config plus a pile of conclusions drawn from it.
Re: industry move that matters for local users - southpatch - 09-26-2026
Quote:A couple things I'd check first. Before rebuilding anything, I'd test one clean baseline with Mistral and compare from there. A lot of these threads e
My first guess would be this: Start by verifying the driver/runtime combo, then test with a smaller prompt, then look at KV cache pressure. Not glamorous advice, but repeatable baselines save a lot of time.
Re: industry move that matters for local users - vectorpoint - 09-26-2026
That matches what I was seeing. That matches what I saw on 7900 XTX.