when RAG was overkill on 5090

[northgarden37]

I figured I'd ask here before changing everything.

My current setup is PyTorch with GB200, and I keep running into VRAM headroom.

Not looking for the perfect answer, mostly what worked in real use.

This is mostly for local work rather than public serving.

[bytesupply76]

Not sure I agree there. The reported numbers can look compelling, but in ordinary usage I didn't see the same benefit. Worth testing the simpler explanation before adopting the more interesting one.

[cacheplane98]

I'd approach it in a boring order. Before rebuilding anything, I'd test one clean baseline with FLUX and compare from there.

[rasterpoint]

Interesting, I hadn't tried that. I ended up in about the same place using SGLang. Worth rechecking your defaults though, because one bad assumption can make the rest of the thread look confusing.

[cozysmith]

I'd approach it in a boring order. Before rebuilding anything, I'd test one clean baseline with Kimi and compare from there.

[sanescope]

I'd start with the simpler stack.

[dockridge]

Thanks, that helps. I don't think your conclusion is far off.

[paperport64]

Tried to reproduce it just now. My result was mixed: first run looked bad, second run looked normal. The main thing that moved was latency, not quality.

[circuitwire58]

A couple things I'd check first. On 5090, the easiest win is often to back off the aggressive settings and see whether the problem survives. Once you know the plain path works, then add the faster path back in piece by piece.

[kernelghost]

My first guess would be this: Before rebuilding anything, I'd test one clean baseline with DeepSeek and compare from there. Once you know the plain path works, then add the faster path back in piece by piece.