+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Optimization & Performance (https://www.datasculpting.ai/inference/forumdisplay.php?fid=10)
+--- Forum: Context / KV Cache (https://www.datasculpting.ai/inference/forumdisplay.php?fid=41)
+--- Thread: draft model choices (/showthread.php?tid=1166)
Pages:
1
2
Re: draft model choices - cacheline - 10-18-2026
subscribed, curious where this lands.
Re: draft model choices - northhall88 - 10-18-2026
following this because I'm in the same boat.
Re: draft model choices - cacheline - 10-18-2026
Quote:What exact setup are you running? Is this local-only or are you routing anything externally?
Appreciate the detail. That matches what I saw on 4090. The biggest change for me was simplifying the stack instead of adding one more optimization layer.
Re: draft model choices - vectorshed43 - 10-18-2026
Quote:following this because I'm in the same boat.
I hit almost the same issue last night.
Re: draft model choices - sparrowcraft - 10-18-2026
A couple questions before I guess too hard. Is this local-only or are you routing anything externally?