+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Miscellaneous Discussion (https://www.datasculpting.ai/inference/forumdisplay.php?fid=24)
+--- Forum: General (https://www.datasculpting.ai/inference/forumdisplay.php?fid=25)
+--- Thread: what made local AI click for you (/showthread.php?tid=330)
Pages:
1
2
Re: what made local AI click for you - quiettoken - 09-22-2026
Can you fill in a bit more context? Are you testing this on A6000 or something else?
Re: what made local AI click for you - ggufwire22 - 09-23-2026
following this because I'm in the same boat.
Re: what made local AI click for you - macroforge - 09-23-2026
following this because I'm in the same boat.
Re: what made local AI click for you - amberrunner - 09-23-2026
subscribed, curious where this lands.
Re: what made local AI click for you - tensorkite - 09-26-2026
Did a small comparison this afternoon. The issue went away once I changed one setting and restarted the stack cleanly.
Re: what made local AI click for you - vectorshed43 - 09-26-2026
Quote:Did a small comparison this afternoon. The issue went away once I changed one setting and restarted the stack cleanly.
What worked for me was pretty simple. Start by verifying the driver/runtime combo, then test with a smaller prompt, then look at tail latency. A lot of these threads end up being one bad config plus a pile of conclusions drawn from it.