+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Training & Customization (https://www.datasculpting.ai/inference/forumdisplay.php?fid=13)
+--- Forum: Distillation (https://www.datasculpting.ai/inference/forumdisplay.php?fid=55)
+--- Thread: when RAG was overkill on 5090 (/showthread.php?tid=929)
Pages:
1
2
when RAG was overkill on 5090 - northgarden37 - 10-09-2026
I figured I'd ask here before changing everything.
My current setup is PyTorch with GB200, and I keep running into VRAM headroom.
Not looking for the perfect answer, mostly what worked in real use.
This is mostly for local work rather than public serving.
Re: when RAG was overkill on 5090 - bytesupply76 - 10-09-2026
Not sure I agree there. The reported numbers can look compelling, but in ordinary usage I didn't see the same benefit. Worth testing the simpler explanation before adopting the more interesting one.
Re: when RAG was overkill on 5090 - cacheplane98 - 10-09-2026
I'd approach it in a boring order. Before rebuilding anything, I'd test one clean baseline with FLUX and compare from there.
Re: when RAG was overkill on 5090 - rasterpoint - 10-09-2026
Interesting, I hadn't tried that. I ended up in about the same place using SGLang. Worth rechecking your defaults though, because one bad assumption can make the rest of the thread look confusing.
Re: when RAG was overkill on 5090 - cozysmith - 10-09-2026
I'd approach it in a boring order. Before rebuilding anything, I'd test one clean baseline with Kimi and compare from there.
Re: when RAG was overkill on 5090 - sanescope - 10-09-2026
I'd start with the simpler stack.
Re: when RAG was overkill on 5090 - dockridge - 10-09-2026
Thanks, that helps. I don't think your conclusion is far off.
Re: when RAG was overkill on 5090 - paperport64 - 10-11-2026
Tried to reproduce it just now. My result was mixed: first run looked bad, second run looked normal. The main thing that moved was latency, not quality.
Re: when RAG was overkill on 5090 - circuitwire58 - 10-11-2026
A couple things I'd check first. On 5090, the easiest win is often to back off the aggressive settings and see whether the problem survives. Once you know the plain path works, then add the faster path back in piece by piece.
Re: when RAG was overkill on 5090 - kernelghost - 10-12-2026
My first guess would be this: Before rebuilding anything, I'd test one clean baseline with DeepSeek and compare from there. Once you know the plain path works, then add the faster path back in piece by piece.