do small edge boxes make sense with llama.cpp

[northgarden37]

I'm trying to sanity-check my setup.

I'm trying to decide whether to keep the current stack or change course. Main question is how people here are handling batch size.

Not looking for the perfect answer, mostly what worked in real use.