+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Generative AI (https://www.datasculpting.ai/inference/forumdisplay.php?fid=17)
+--- Forum: Video (https://www.datasculpting.ai/inference/forumdisplay.php?fid=70)
+--- Thread: multimodal local stack that feels real (/showthread.php?tid=1061)
Pages:
1
2
Re: multimodal local stack that feels real - visionforge - 10-18-2026
Appreciate the detail. The pattern around long context has been pretty similar for me. The biggest change for me was simplifying the stack instead of adding one more optimization layer.
Re: multimodal local stack that feels real - audiopath66 - 10-18-2026
Can you fill in a bit more context? Do you see the same thing with a smaller batch or shorter prompt?
Re: multimodal local stack that feels real - oxideworks59 - 10-19-2026
What exact setup are you running? Are you testing this on 7900 XTX or something else?
Re: multimodal local stack that feels real - paperrunner - 10-19-2026
I tested this tonight. On GB200 with TensorRT-LLM, I saw basically the same behavior. I expected the model choice to matter more than it did.
Re: multimodal local stack that feels real - forgemixer - 10-19-2026
Quote:Did a small comparison this afternoon. On B200 with SGLang, I saw basically the same behavior.
A couple questions before I guess too hard. Is this local-only or are you routing anything externally?
Re: multimodal local stack that feels real - latentpeak22 - 10-19-2026
Did a small comparison this afternoon. On B200 with SGLang, I saw basically the same behavior.