simple local serving recipe: anyone testing this?

[cudapoint]

Posting this in case it saves someone else a little time.

I'm helping a small team set up a local workflow. We don't need a giant stack, but we do care about tail latency.

I'm especially interested in mistakes worth avoiding.

For context, I'm not chasing leaderboard numbers. I mostly want the system to feel consistent day to day.