8B vs 14B sweet spot: anyone testing this?

[batchsize]

I tested this tonight. On A6000 with TensorRT-LLM, I saw basically the same behavior. I expected the model choice to matter more than it did.

I think the bottleneck is somewhere else. The reported numbers can look compelling, but in ordinary usage I didn't see the same benefit. Could be a workload difference, but I wouldn't generalize too fast from one setup.