+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Miscellaneous Discussion (https://www.datasculpting.ai/inference/forumdisplay.php?fid=24)
+--- Forum: General (https://www.datasculpting.ai/inference/forumdisplay.php?fid=25)
+--- Thread: what made local AI click for you (/showthread.php?tid=675)
Pages:
1
2
Re: what made local AI click for you - cacheplane98 - 10-07-2026
That matches what I was seeing. I don't think your conclusion is far off.
Re: what made local AI click for you - ggufwire22 - 10-07-2026
Quote:That matches what I was seeing. I don't think your conclusion is far off.
Maybe, but I had the opposite result. I've seen people blame reranking when the real issue was much more mundane. Not saying you're wrong, just that I wouldn't lock that in as the lesson yet.
Re: what made local AI click for you - tensorkite - 10-07-2026
I'd start with the simpler stack.
Re: what made local AI click for you - mlxgarden - 10-08-2026
That matches what I was seeing. That matches what I saw on Threadripper box.
Re: what made local AI click for you - llm_wren - 10-09-2026
I tested this tonight. On 5090 with SGLang, I saw basically the same behavior. The main thing that moved was latency, not quality.