+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Hardware (https://www.datasculpting.ai/inference/forumdisplay.php?fid=4)
+--- Forum: GPUs (https://www.datasculpting.ai/inference/forumdisplay.php?fid=28)
+--- Thread: dual GPU split inference for Mistral (/showthread.php?tid=63)
Pages:
1
2
Re: dual GPU split inference for Mistral - circuitwire58 - 09-07-2026
Can you fill in a bit more context? Is this local-only or are you routing anything externally?
Re: dual GPU split inference for Mistral - dockkite69 - 09-07-2026
Thanks, that helps. The pattern around structured output has been pretty similar for me. The biggest change for me was simplifying the stack instead of adding one more optimization layer. The logs were somehow calmer than I was.
Re: dual GPU split inference for Mistral - vrambandit - 09-08-2026
same here on my side.
Re: dual GPU split inference for Mistral - rocmriley - 09-08-2026
Quote:same here on my side.
same here on my side.
Re: dual GPU split inference for Mistral - paperrunner - 09-09-2026
Quote:Quote:same here on my side.
That matches what I was seeing. The pattern around prefix caching has been pretty similar for me. The biggest change for me was simplifying the stack instead of adding one more optimization layer.