+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Edge AI & Gadgets (https://www.datasculpting.ai/inference/forumdisplay.php?fid=21)
+--- Forum: AI Devices (https://www.datasculpting.ai/inference/forumdisplay.php?fid=75)
+--- Thread: odd little inference appliances: early impressions (/showthread.php?tid=384)
Pages:
1
2
Re: odd little inference appliances: early impressions - turboscope - 09-28-2026
A couple questions before I guess too hard. Is this local-only or are you routing anything externally?
Re: odd little inference appliances: early impressions - routeline23 - 09-30-2026
Quote:A couple questions before I guess too hard. Is this local-only or are you routing anything externally?
subscribed, curious where this lands.
Re: odd little inference appliances: early impressions - turbosupply83 - 09-30-2026
I tested this tonight. I couldn't reproduce the slowdown until I pushed context much higher. The main thing that moved was latency, not quality.
Re: odd little inference appliances: early impressions - macroforge - 09-30-2026
Tried to reproduce it just now. The issue went away once I changed one setting and restarted the stack cleanly. The main thing that moved was latency, not quality.
Re: odd little inference appliances: early impressions - routeline23 - 09-30-2026
Appreciate the detail. That matches what I saw on 5090. Worth rechecking your defaults though, because one bad assumption can make the rest of the thread look confusing.
Re: odd little inference appliances: early impressions - mangopath4 - 09-30-2026
worth checking the driver first.