QLoRA resource reality check on Threadripper box

[kernelpeak]

Posting this in case it saves someone else a little time.

If you were starting from scratch today for this use case, what would you choose first and what would you ignore?

I'm especially interested in mistakes worth avoiding.

[batchsize]

Posting this in case it saves someone else a little time.

Thanks, that helps. I don't think your conclusion is far off. Worth rechecking your defaults though, because one bad assumption can make the rest of the thread look confusing.