Help with Embeddings

[torchdesk]

Posting this in case it saves someone else a little time.

If you were starting from scratch today for this use case, what would you choose first and what would you ignore?

Would rather learn from your scars than collect my own all at once.

For context, I'm not chasing leaderboard numbers. I mostly want the system to feel consistent day to day.

[quantnest]

My first guess would be this: On 4090, the easiest win is often to back off the aggressive settings and see whether the problem survives. A lot of these threads end up being one bad config plus a pile of conclusions drawn from it.

[paperport64]

Thanks, that helps. The pattern around small local stacks has been pretty similar for me. The biggest change for me was simplifying the stack instead of adding one more optimization layer.

[edgecoast]

Appreciate the detail. I don't think your conclusion is far off. I got better results once I stopped chasing one metric and just looked at whether the workflow felt usable.

If you post your exact settings, people here can probably narrow it down pretty fast.

[vectorshed43]

I'd start with the simpler stack.

[edgecoast]

What exact setup are you running? Do you see the same thing with a smaller batch or shorter prompt?

I'm happy to test one more variant if you want a second data point.

[quantnest]

Thanks, that helps. I ended up in about the same place using PyTorch. The biggest change for me was simplifying the stack instead of adding one more optimization layer.

[vramscope]

A couple things I'd check first. On 4090, the easiest win is often to back off the aggressive settings and see whether the problem survives.

[rasterhall]

A couple questions before I guess too hard. Do you see the same thing with a smaller batch or shorter prompt?

[vramscope]

subscribed, curious where this lands.