+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Training & Customization (https://www.datasculpting.ai/inference/forumdisplay.php?fid=13)
+--- Forum: Distillation (https://www.datasculpting.ai/inference/forumdisplay.php?fid=55)
+--- Thread: when RAG was overkill for DeepSeek (/showthread.php?tid=433)
when RAG was overkill for DeepSeek - quantshed - 09-23-2026
This keeps coming up in conversation so I figured I'd start a thread.
The more I work with local setups, the more it seems like prefill/decode split decides everything downstream.
Curious whether that lines up with what other people are seeing.
Re: when RAG was overkill for DeepSeek - batchyard - 09-25-2026
Can you fill in a bit more context? Are you testing this on 7900 XTX or something else?
Re: when RAG was overkill for DeepSeek - modrunner27 - 09-26-2026
I think the bottleneck is somewhere else. I've seen people blame tail latency when the real issue was much more mundane. Not saying you're wrong, just that I wouldn't lock that in as the lesson yet.
Re: when RAG was overkill for DeepSeek - rasterbench - 09-26-2026
Can you fill in a bit more context? What model, what runtime, and roughly what context size?
Re: when RAG was overkill for DeepSeek - routecraft - 09-26-2026
My first guess would be this: Start by verifying the driver/runtime combo, then test with a smaller prompt, then look at speculative decoding. A lot of these threads end up being one bad config plus a pile of conclusions drawn from it.
Re: when RAG was overkill for DeepSeek - mangotinker73 - 09-26-2026
Quote:My first guess would be this: Start by verifying the driver/runtime combo, then test with a smaller prompt, then look at speculative decoding. A lot o
Ran a quick check on my box. I couldn't reproduce the slowdown until I pushed context much higher. The main thing that moved was latency, not quality.
Re: when RAG was overkill for DeepSeek - amberpatch25 - 09-27-2026
My first guess would be this: If you're on SGLang, I'd confirm the version first and then reduce variables until the behavior is obvious. A lot of these threads end up being one bad config plus a pile of conclusions drawn from it.