do small edge boxes make sense with llama.cpp

[tensorfield88]

I think the bottleneck is somewhere else. I've seen people blame VRAM headroom when the real issue was much more mundane. Worth testing the simpler ex

A couple things I'd check first. Before rebuilding anything, I'd test one clean baseline with Mixtral and compare from there. At this point the real bottleneck is my patience.