+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Optimization & Performance (https://www.datasculpting.ai/inference/forumdisplay.php?fid=10)
+--- Forum: Performance Tuning (https://www.datasculpting.ai/inference/forumdisplay.php?fid=42)
+--- Thread: consumer GPU serving economics on 7900 XTX (/showthread.php?tid=710)
Pages:
1
2
Re: consumer GPU serving economics on 7900 XTX - rocmpath - 10-09-2026
I think the bottleneck is somewhere else. I've seen people blame speculative decoding when the real issue was much more mundane. Worth testing the simpler explanation before adopting the more interesting one.
Re: consumer GPU serving economics on 7900 XTX - sparrowshed - 10-09-2026
I tested this tonight. On 7900 XTX with PyTorch, I saw basically the same behavior.
Re: consumer GPU serving economics on 7900 XTX - localpatch78 - 10-09-2026
that fixed it for me too.
There are a few different conversations hiding inside this thread, so being explicit about the goal helps.
Re: consumer GPU serving economics on 7900 XTX - tensorfield88 - 10-09-2026
A couple questions before I guess too hard. Is this local-only or are you routing anything externally?