+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Hardware (https://www.datasculpting.ai/inference/forumdisplay.php?fid=4)
+--- Forum: Systems & Builds (https://www.datasculpting.ai/inference/forumdisplay.php?fid=29)
+--- Thread: power delivery for 24/7 serving (/showthread.php?tid=325)
power delivery for 24/7 serving - tensorworks - 09-17-2026
Did a bit of testing and thought I'd post the result here.
I tried this with Qwen using vLLM on 7900 XTX. The headline result was better than I expected, but NVFP4 still ended up being the limiting factor.
Happy to post more details if anyone wants them.
This is mostly for local work rather than public serving.
Re: power delivery for 24/7 serving - batchwire - 09-19-2026
That matches what I was seeing. The pattern around prefill/decode split has been pretty similar for me.
Re: power delivery for 24/7 serving - batchyard - 09-19-2026
A couple things I'd check first. Start by verifying the driver/runtime combo, then test with a smaller prompt, then look at prefix caching. Not glamorous advice, but repeatable baselines save a lot of time.
Re: power delivery for 24/7 serving - promptspark58 - 09-20-2026
My first guess would be this: Before rebuilding anything, I'd test one clean baseline with Kimi and compare from there. Once you know the plain path works, then add the faster path back in piece by piece.
Re: power delivery for 24/7 serving - edgequill - 09-20-2026
I tested this tonight. The issue went away once I changed one setting and restarted the stack cleanly. The surprise was that memory pressure showed up before throughput collapsed.
Re: power delivery for 24/7 serving - embershed99 - 09-20-2026
A couple things I'd check first. If you're on SGLang, I'd confirm the version first and then reduce variables until the behavior is obvious. Not glamorous advice, but repeatable baselines save a lot of time.
Re: power delivery for 24/7 serving - batchwire - 09-20-2026
I think the bottleneck is somewhere else. I get why that seems right, but on 5090 the bottleneck felt elsewhere.