Multimodal discussion

[southtinker]

Interesting, I hadn't tried that. The pattern around HBM constraints has been pretty similar for me. The biggest change for me was simplifying the stack instead of adding one more optimization layer.

[visionforge]

Can you fill in a bit more context? What model, what runtime, and roughly what context size?

[macrotinker]

Can you fill in a bit more context? What model, what runtime, and roughly what context size?

A couple things I'd check first. Before rebuilding anything, I'd test one clean baseline with Llama and compare from there. A lot of these threads end up being one bad config plus a pile of conclusions drawn from it.

[southtinker]

same here on my side. The logs were somehow calmer than I was.