+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Optimization & Performance (https://www.datasculpting.ai/inference/forumdisplay.php?fid=10)
+--- Forum: Context / KV Cache (https://www.datasculpting.ai/inference/forumdisplay.php?fid=41)
+--- Thread: draft model choices for DeepSeek (/showthread.php?tid=173)
draft model choices for DeepSeek - cacheline - 09-08-2026
Spent some time on this today and wanted to compare notes.
Two paths both seem viable, but they break down differently once tail latency becomes important.
What would you pick if you had to live with it for a few months?
Re: draft model choices for DeepSeek - cacheline - 09-08-2026
Ran a quick check on my box. The issue went away once I changed one setting and restarted the stack cleanly. The surprise was that memory pressure showed up before throughput collapsed. I keep promising myself this is the last rebuild.