eval setup for local models on M3 Max

[griffinloom66]

Spent some time on this today and wanted to compare notes.

I'm comparing two approaches here: keep it simple on 5090, or squeeze more out of the stack and deal with the complexity.

Trying to avoid rebuilding the whole thing twice.

This is mostly for local work rather than public serving.

[cacheline]

Appreciate the detail. That matches what I saw on 5090. I got better results once I stopped chasing one metric and just looked at whether the workflow felt usable.