PCIe lane weirdness with inference loads: early impressions

[turboyard94]

Thought I'd post a small recipe because a lot of threads ask similar things.

This isn't revolutionary, but the combination of PyTorch and mini PC felt surprisingly usable once I tuned around tail latency.

Feel free to tear it apart if you see something dumb.

This is mostly for local work rather than public serving.