latency vs throughput for small deployments: setup advice

[macropilot]

This keeps coming up in conversation so I figured I'd start a thread.

The more I work with local setups, the more it seems like token cost decides everything downstream.

Wondering if this is just where the whole space is heading now.