+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Generative AI (https://www.datasculpting.ai/inference/forumdisplay.php?fid=17)
+--- Forum: Multimodal (https://www.datasculpting.ai/inference/forumdisplay.php?fid=72)
+--- Thread: Multimodal discussion (/showthread.php?tid=422)
Pages:
1
2
Multimodal discussion - visionforge - 09-22-2026
Did a bit of testing and thought I'd post the result here.
After a few runs on 7900 XTX, I ended up with decent performance, but the setup only really made sense once I adjusted around quant drift.
Happy to post more details if anyone wants them.
I care more about stability than squeezing the last possible tokens/sec out of it.
Re: Multimodal discussion - lablab - 09-24-2026
What worked for me was pretty simple. Before rebuilding anything, I'd test one clean baseline with DeepSeek and compare from there. A lot of these threads end up being one bad config plus a pile of conclusions drawn from it.
Re: Multimodal discussion - vramarch - 09-25-2026
I don't think that's the whole story. I wouldn't optimize around that first. On my side the bigger swing came from the runtime and not the model. Not saying you're wrong, just that I wouldn't lock that in as the lesson yet.
Re: Multimodal discussion - macrostack69 - 09-25-2026
What worked for me was pretty simple. Before rebuilding anything, I'd test one clean baseline with Llama and compare from there. Once you know the plain path works, then add the faster path back in piece by piece.
Re: Multimodal discussion - macropilot - 09-26-2026
I think the bottleneck is somewhere else. I wouldn't optimize around that first. On my side the bigger swing came from the runtime and not the model.
Re: Multimodal discussion - fastquill74 - 09-26-2026
Appreciate the detail. I don't think your conclusion is far off. Worth rechecking your defaults though, because one bad assumption can make the rest of the thread look confusing.
Re: Multimodal discussion - macrostack69 - 09-27-2026
I'd approach it in a boring order. Start by verifying the driver/runtime combo, then test with a smaller prompt, then look at VRAM headroom. Once you know the plain path works, then add the faster path back in piece by piece.
Re: Multimodal discussion - torchshed66 - 09-27-2026
Quote:I'd approach it in a boring order. Start by verifying the driver/runtime combo, then test with a smaller prompt, then look at VRAM headroom. Once you
I don't think that's the whole story. I get why that seems right, but on mini PC the bottleneck felt elsewhere. Not saying you're wrong, just that I wouldn't lock that in as the lesson yet.
There are a few different conversations hiding inside this thread, so being explicit about the goal helps.
Re: Multimodal discussion - macropilot - 09-27-2026
Interesting, I hadn't tried that. That matches what I saw on Threadripper box.
Re: Multimodal discussion - mangovault67 - 09-27-2026
Appreciate the detail. The pattern around reranking has been pretty similar for me. The biggest change for me was simplifying the stack instead of adding one more optimization layer.