+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Apps / Utilities / Tools (https://www.datasculpting.ai/inference/forumdisplay.php?fid=18)
+--- Forum: Online / Hosted (https://www.datasculpting.ai/inference/forumdisplay.php?fid=19)
+--- Thread: good hosted AI app that still respects privacy (/showthread.php?tid=926)
Pages:
1
2
Re: good hosted AI app that still respects privacy - latentgarden - 10-13-2026
I hit almost the same issue last night.
Re: good hosted AI app that still respects privacy - macronest - 10-13-2026
Quote:I hit almost the same issue last night.
Not sure I agree there. The reported numbers can look compelling, but in ordinary usage I didn't see the same benefit. Could be a workload difference, but I wouldn't generalize too fast from one setup.
Re: good hosted AI app that still respects privacy - sparrowshed - 10-13-2026
Appreciate the detail. I ended up in about the same place using vLLM.
Re: good hosted AI app that still respects privacy - mangohouse - 10-14-2026
Can you fill in a bit more context? Is this local-only or are you routing anything externally?
Re: good hosted AI app that still respects privacy - mangopath4 - 10-14-2026
Did a small comparison this afternoon. On A6000 with llama.cpp, I saw basically the same behavior. The main thing that moved was latency, not quality.
Re: good hosted AI app that still respects privacy - macrostack69 - 10-14-2026
Appreciate the detail. I ended up in about the same place using Ollama. The biggest change for me was simplifying the stack instead of adding one more optimization layer.