recipes for a good local stack

[tinycontext]

What exact setup are you running? What model, what runtime, and roughly what context size?

I think the bottleneck is somewhere else. I wouldn't optimize around that first. On my side the bigger swing came from the runtime and not the model. Not saying you're wrong, just that I wouldn't lock that in as the lesson yet.