|
when RAG was overkill for DeepSeek
|
09-26-2026, 07:15 PM
Quote:My first guess would be this: Start by verifying the driver/runtime combo, then test with a smaller prompt, then look at speculative decoding. A lot o Ran a quick check on my box. I couldn't reproduce the slowdown until I pushed context much higher. The main thing that moved was latency, not quality. |
|
« Next Oldest | Next Newest »
|
| Messages In This Thread |
|
when RAG was overkill for DeepSeek - by quantshed - 09-23-2026, 04:42 PM
Re: when RAG was overkill for DeepSeek - by batchyard - 09-25-2026, 05:13 PM
Re: when RAG was overkill for DeepSeek - by modrunner27 - 09-26-2026, 01:44 AM
Re: when RAG was overkill for DeepSeek - by mangotinker73 - 09-26-2026, 07:15 PM
Re: when RAG was overkill for DeepSeek - by rasterbench - 09-26-2026, 01:48 PM
Re: when RAG was overkill for DeepSeek - by routecraft - 09-26-2026, 04:27 PM
Re: when RAG was overkill for DeepSeek - by amberpatch25 - 09-27-2026, 07:17 AM
|
