first LoRA that actually helped

[cacheline]

Posting this in case it saves someone else a little time.

If you were starting from scratch today for this use case, what would you choose first and what would you ignore?

I'm especially interested in mistakes worth avoiding.

This is mostly for local work rather than public serving.

[benchplease]

A couple questions before I guess too hard. Is this local-only or are you routing anything externally?

[mangohouse]

A couple questions before I guess too hard. Are you testing this on 4090 or something else?

[batchsize]

I think the bottleneck is somewhere else. I've seen people blame HBM constraints when the real issue was much more mundane. Worth testing the simpler explanation before adopting the more interesting one.

I'm happy to test one more variant if you want a second data point.

[griffinloom66]

I think the bottleneck is somewhere else. I've seen people blame HBM constraints when the real issue was much more mundane. Worth testing the simpler

Appreciate the detail. That matches what I saw on 5090. I got better results once I stopped chasing one metric and just looked at whether the workflow felt usable.

[northgrid33]

Appreciate the detail. I don't think your conclusion is far off.

[routecraft]

A couple questions before I guess too hard. Do you see the same thing with a smaller batch or shorter prompt?

[batchsize]

Good point. I don't think your conclusion is far off. I got better results once I stopped chasing one metric and just looked at whether the workflow felt usable.

[batchpeak]

A couple things I'd check first. Before rebuilding anything, I'd test one clean baseline with DeepSeek and compare from there. A lot of these threads end up being one bad config plus a pile of conclusions drawn from it.

[batchsize]

Thanks, that helps. That matches what I saw on B200. Worth rechecking your defaults though, because one bad assumption can make the rest of the thread look confusing.