+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Apps / Utilities / Tools (https://www.datasculpting.ai/inference/forumdisplay.php?fid=18)
+--- Forum: Local (https://www.datasculpting.ai/inference/forumdisplay.php?fid=20)
+--- Thread: desktop apps around local AI: setup advice (/showthread.php?tid=508)
Pages:
1
2
desktop apps around local AI: setup advice - rocmriley - 09-27-2026
Spent some time on this today and wanted to compare notes.
I'm comparing two approaches here: keep it simple on Threadripper box, or squeeze more out of the stack and deal with the complexity.
Interested in tradeoffs more than fan loyalty.
Re: desktop apps around local AI: setup advice - vrampilot - 09-28-2026
Ran a quick check on my box. I couldn't reproduce the slowdown until I pushed context much higher. I expected the model choice to matter more than it did.
Re: desktop apps around local AI: setup advice - cacheworks - 09-28-2026
Interesting, I hadn't tried that. I don't think your conclusion is far off. The biggest change for me was simplifying the stack instead of adding one more optimization layer.
Re: desktop apps around local AI: setup advice - rocmriley - 09-28-2026
Quote:Interesting, I hadn't tried that. I don't think your conclusion is far off. The biggest change for me was simplifying the stack instead of adding one
following this because I'm in the same boat.
Re: desktop apps around local AI: setup advice - mlxgarden - 09-29-2026
My first guess would be this: Before rebuilding anything, I'd test one clean baseline with FLUX and compare from there.
Re: desktop apps around local AI: setup advice - vramstream - 09-29-2026
Can you fill in a bit more context? What model, what runtime, and roughly what context size?
Re: desktop apps around local AI: setup advice - cachebench - 10-02-2026
A couple things I'd check first. If you're on LM Studio, I'd confirm the version first and then reduce variables until the behavior is obvious. A lot of these threads end up being one bad config plus a pile of conclusions drawn from it. Turns out 'just one more test' was a lie.
Re: desktop apps around local AI: setup advice - bytegarden73 - 10-05-2026
Thanks, that helps. I ended up in about the same place using TensorRT-LLM. The biggest change for me was simplifying the stack instead of adding one more optimization layer.
Re: desktop apps around local AI: setup advice - mlxgarden - 10-06-2026
A couple things I'd check first. If you're on PyTorch, I'd confirm the version first and then reduce variables until the behavior is obvious.
Re: desktop apps around local AI: setup advice - rocmriley - 10-06-2026
I'd approach it in a boring order. If you're on Ollama, I'd confirm the version first and then reduce variables until the behavior is obvious. A lot of these threads end up being one bad config plus a pile of conclusions drawn from it. At this point the real bottleneck is my patience.