+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Hardware (https://www.datasculpting.ai/inference/forumdisplay.php?fid=4)
+--- Forum: GPUs (https://www.datasculpting.ai/inference/forumdisplay.php?fid=28)
+--- Thread: dual GPU split inference (/showthread.php?tid=95)
dual GPU split inference - routecrate - 09-02-2026
Posting this in case it saves someone else a little time.
I'm helping a small team set up a local workflow. We don't need a giant stack, but we do care about prefill/decode split.
Would rather learn from your scars than collect my own all at once.
Re: dual GPU split inference - amberhall - 09-03-2026
Tried to reproduce it just now. I couldn't reproduce the slowdown until I pushed context much higher. The main thing that moved was latency, not quality.
Re: dual GPU split inference - cudapoint - 09-03-2026
I tested this tonight. My result was mixed: first run looked bad, second run looked normal. I expected the model choice to matter more than it did.