Multimodal discussion: setup advice

[torchsupply99]

I figured I'd ask here before changing everything.

I'm trying to decide whether to keep the current stack or change course. Main question is how people here are handling tokens/sec.

If you've tested something similar, what did you keep and what did you stop bothering with?

[macropilot]

Tried to reproduce it just now. I couldn't reproduce the slowdown until I pushed context much higher.