+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Apps / Utilities / Tools (https://www.datasculpting.ai/inference/forumdisplay.php?fid=18)
+--- Forum: Online / Hosted (https://www.datasculpting.ai/inference/forumdisplay.php?fid=19)
+--- Thread: good hosted AI app that still respects privacy on A6000 (/showthread.php?tid=949)
Pages:
1
2
good hosted AI app that still respects privacy on A6000 - torchsmith25 - 10-09-2026
Did a bit of testing and thought I'd post the result here.
I tried this with Mistral using LM Studio on A6000. The headline result was better than I expected, but INT4 still ended up being the limiting factor.
Happy to post more details if anyone wants them.
Re: good hosted AI app that still respects privacy on A6000 - deepseeked - 10-09-2026
A couple things I'd check first. Start by verifying the driver/runtime combo, then test with a smaller prompt, then look at quant drift.
Re: good hosted AI app that still respects privacy on A6000 - audiodesk - 10-09-2026
Thanks, that helps. The pattern around structured output has been pretty similar for me. Worth rechecking your defaults though, because one bad assumption can make the rest of the thread look confusing.
There are a few different conversations hiding inside this thread, so being explicit about the goal helps.
Re: good hosted AI app that still respects privacy on A6000 - mintstation99 - 10-09-2026
My first guess would be this: Start by verifying the driver/runtime combo, then test with a smaller prompt, then look at router policy. A lot of these threads end up being one bad config plus a pile of conclusions drawn from it.
Re: good hosted AI app that still respects privacy on A6000 - emberpeak - 10-10-2026
Interesting, I hadn't tried that. That matches what I saw on Threadripper box. I keep promising myself this is the last rebuild.
Re: good hosted AI app that still respects privacy on A6000 - cachebench - 10-12-2026
Can you fill in a bit more context? Are you testing this on 5090 or something else?
Re: good hosted AI app that still respects privacy on A6000 - latentgarden - 10-12-2026
Quote:Can you fill in a bit more context? Are you testing this on 5090 or something else?
Did a small comparison this afternoon. The issue went away once I changed one setting and restarted the stack cleanly. I expected the model choice to matter more than it did.
Re: good hosted AI app that still respects privacy on A6000 - batchshed - 10-14-2026
That matches what I was seeing. That matches what I saw on 5090.
Re: good hosted AI app that still respects privacy on A6000 - quantghost - 10-14-2026
Good point. That matches what I saw on mini PC. The biggest change for me was simplifying the stack instead of adding one more optimization layer.
Re: good hosted AI app that still respects privacy on A6000 - amberplane - 10-14-2026
Quote:Good point. That matches what I saw on mini PC. The biggest change for me was simplifying the stack instead of adding one more optimization layer.
My first guess would be this: On mini PC, the easiest win is often to back off the aggressive settings and see whether the problem survives.