+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Miscellaneous Discussion (https://www.datasculpting.ai/inference/forumdisplay.php?fid=24)
+--- Forum: News & Industry (https://www.datasculpting.ai/inference/forumdisplay.php?fid=26)
+--- Thread: industry move that matters for local users (/showthread.php?tid=1114)
Pages:
1
2
industry move that matters for local users - deepseeked - 10-12-2026
Did a bit of testing and thought I'd post the result here.
After a few runs on A6000, I ended up with decent performance, but the setup only really made sense once I adjusted around consumer GPUs.
Curious if anyone else landed in the same place.
Re: industry move that matters for local users - southpatch - 10-13-2026
Thanks, that helps. The pattern around VRAM headroom has been pretty similar for me. I got better results once I stopped chasing one metric and just looked at whether the workflow felt usable.
Re: industry move that matters for local users - kernelmike - 10-13-2026
A couple things I'd check first. If you're on Ollama, I'd confirm the version first and then reduce variables until the behavior is obvious. Not glamorous advice, but repeatable baselines save a lot of time.
Re: industry move that matters for local users - deepseeked - 10-13-2026
Quote:A couple things I'd check first. If you're on Ollama, I'd confirm the version first and then reduce variables until the behavior is obvious. Not glamo
A couple things I'd check first. On M3 Max, the easiest win is often to back off the aggressive settings and see whether the problem survives. I keep promising myself this is the last rebuild.
Re: industry move that matters for local users - phaseshed - 10-14-2026
Thanks, that helps. I don't think your conclusion is far off.
Re: industry move that matters for local users - bytelab - 10-17-2026
A couple things I'd check first. On 5090, the easiest win is often to back off the aggressive settings and see whether the problem survives. Once you know the plain path works, then add the faster path back in piece by piece.
Re: industry move that matters for local users - modbench - 10-18-2026
I think the bottleneck is somewhere else. I get why that seems right, but on dual 3090 the bottleneck felt elsewhere. Not saying you're wrong, just that I wouldn't lock that in as the lesson yet.
Re: industry move that matters for local users - dockriver - 10-18-2026
I'd approach it in a boring order. Before rebuilding anything, I'd test one clean baseline with Llama and compare from there. Once you know the plain path works, then add the faster path back in piece by piece.
Re: industry move that matters for local users - fastlab - 10-18-2026
Good point. I ended up in about the same place using Ollama. I got better results once I stopped chasing one metric and just looked at whether the workflow felt usable.
Re: industry move that matters for local users - bytestack - 10-18-2026
I tested this tonight. The issue went away once I changed one setting and restarted the stack cleanly.