+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Optimization & Performance (https://www.datasculpting.ai/inference/forumdisplay.php?fid=10)
+--- Forum: Context / KV Cache (https://www.datasculpting.ai/inference/forumdisplay.php?fid=41)
+--- Thread: draft model choices (/showthread.php?tid=492)
Pages:
1
2
Re: draft model choices - cacheline - 09-28-2026
Appreciate the detail. That matches what I saw on Threadripper box. I got better results once I stopped chasing one metric and just looked at whether the workflow felt usable.
Re: draft model choices - bytepilot - 09-28-2026
Appreciate the detail. The pattern around small local stacks has been pretty similar for me. Worth rechecking your defaults though, because one bad assumption can make the rest of the thread look confusing. Turns out 'just one more test' was a lie.