Help with Embeddings for Phi

[vectorshed43]

Posting this in case it saves someone else a little time.

I'm not trying to build a perfect architecture diagram. I just want a sane starting point around LM Studio and 7900 XTX.

Would rather learn from your scars than collect my own all at once.

[torchdesk]

What worked for me was pretty simple. On GB200, the easiest win is often to back off the aggressive settings and see whether the problem survives. Not glamorous advice, but repeatable baselines save a lot of time.

[edgecoast]

That matches what I was seeing. The pattern around batch size has been pretty similar for me. The biggest change for me was simplifying the stack instead of adding one more optimization layer.

[quantnest]

My first guess would be this: On M3 Max, the easiest win is often to back off the aggressive settings and see whether the problem survives. Not glamorous advice, but repeatable baselines save a lot of time.

[batchcraft]

Can you fill in a bit more context? Is this local-only or are you routing anything externally?

[edgequill]

Thanks, that helps. The pattern around consumer GPUs has been pretty similar for me. I got better results once I stopped chasing one metric and just looked at whether the workflow felt usable.

[rasterhall]

I don't think that's the whole story. I get why that seems right, but on Threadripper box the bottleneck felt elsewhere. Worth testing the simpler explanation before adopting the more interesting one.

[paperport64]

Can you fill in a bit more context? What model, what runtime, and roughly what context size?

[ggufwire22]

My first guess would be this: Before rebuilding anything, I'd test one clean baseline with FLUX and compare from there. A lot of these threads end up being one bad config plus a pile of conclusions drawn from it.

[ggufwire22]

Need a little more detail here. What model, what runtime, and roughly what context size?