new here, what's your stack on Threadripper box

[rocmplane]

Posting this in case it saves someone else a little time.

I'm helping a small team set up a local workflow. We don't need a giant stack, but we do care about consumer GPUs.

I'm especially interested in mistakes worth avoiding.

This is mostly for local work rather than public serving.

[ggufwire22]

A couple things I'd check first. Start by verifying the driver/runtime combo, then test with a smaller prompt, then look at small local stacks. A lot of these threads end up being one bad config plus a pile of conclusions drawn from it.