+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Inference Engines (https://www.datasculpting.ai/inference/forumdisplay.php?fid=9)
+--- Forum: llama.cpp / GGUF (https://www.datasculpting.ai/inference/forumdisplay.php?fid=35)
+--- Thread: multi-GPU offload advice (/showthread.php?tid=976)
multi-GPU offload advice - vectorport - 10-10-2026
Spent some time on this today and wanted to compare notes.
I'm comparing two approaches here: keep it simple on GB200, or squeeze more out of the stack and deal with the complexity.
What would you pick if you had to live with it for a few months?
I care more about stability than squeezing the last possible tokens/sec out of it.
Re: multi-GPU offload advice - signalpeak - 10-11-2026
Appreciate the detail. I don't think your conclusion is far off. The biggest change for me was simplifying the stack instead of adding one more optimization layer. Turns out 'just one more test' was a lie.
Re: multi-GPU offload advice - northcraft90 - 10-11-2026
Thanks, that helps. The pattern around prefill/decode split has been pretty similar for me. I got better results once I stopped chasing one metric and just looked at whether the workflow felt usable.