+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Generative AI (https://www.datasculpting.ai/inference/forumdisplay.php?fid=17)
+--- Forum: Multimodal (https://www.datasculpting.ai/inference/forumdisplay.php?fid=72)
+--- Thread: Multimodal discussion (/showthread.php?tid=1290)
Pages:
1
2
Multimodal discussion - visionforge - 10-16-2026
Did a bit of testing and thought I'd post the result here.
I tried this with Kimi using MLX on 4090. The headline result was better than I expected, but KV cache pressure still ended up being the limiting factor.
Curious if anyone else landed in the same place.
For context, I'm not chasing leaderboard numbers. I mostly want the system to feel consistent day to day.
Re: Multimodal discussion - southtinker - 10-16-2026
Quote:Did a bit of testing and thought I'd post the result here.
Can you fill in a bit more context? What model, what runtime, and roughly what context size?
Re: Multimodal discussion - macropilot - 10-16-2026
I'd approach it in a boring order. If you're on vLLM, I'd confirm the version first and then reduce variables until the behavior is obvious.
Re: Multimodal discussion - kernelmike - 10-17-2026
worth checking the driver first.
Re: Multimodal discussion - quietcraft - 10-17-2026
Quote:worth checking the driver first.
subscribed, curious where this lands.
Re: Multimodal discussion - vramscope - 10-17-2026
A couple questions before I guess too hard. Do you see the same thing with a smaller batch or shorter prompt?
Re: Multimodal discussion - mintstream9 - 10-18-2026
What exact setup are you running? Is this local-only or are you routing anything externally? Turns out 'just one more test' was a lie.
Re: Multimodal discussion - batchwire - 10-18-2026
I don't think that's the whole story. I wouldn't optimize around that first. On my side the bigger swing came from the runtime and not the model.
Re: Multimodal discussion - sparrowgrid - 10-18-2026
Maybe, but I had the opposite result. I get why that seems right, but on 7900 XTX the bottleneck felt elsewhere. Could be a workload difference, but I wouldn't generalize too fast from one setup.
Re: Multimodal discussion - torchshed66 - 10-18-2026
I'd approach it in a boring order. On mini PC, the easiest win is often to back off the aggressive settings and see whether the problem survives. Not glamorous advice, but repeatable baselines save a lot of time. The logs were somehow calmer than I was.