+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Miscellaneous Discussion (https://www.datasculpting.ai/inference/forumdisplay.php?fid=24)
+--- Forum: General (https://www.datasculpting.ai/inference/forumdisplay.php?fid=25)
+--- Thread: what are you all running this week on dual 3090 (/showthread.php?tid=124)
Pages:
1
2
Re: what are you all running this week on dual 3090 - rocmrunner40 - 09-08-2026
Quote:I tested this tonight. I couldn't reproduce the slowdown until I pushed context much higher.
That matches what I was seeing. The pattern around prefix caching has been pretty similar for me. I got better results once I stopped chasing one metric and just looked at whether the workflow felt usable.
Re: what are you all running this week on dual 3090 - vectorshed43 - 09-08-2026
I tested this tonight. I couldn't reproduce the slowdown until I pushed context much higher. The main thing that moved was latency, not quality.
Re: what are you all running this week on dual 3090 - plaingarden18 - 09-08-2026
That matches what I was seeing. I don't think your conclusion is far off.
Re: what are you all running this week on dual 3090 - tensorkite - 09-08-2026
Quote:That matches what I was seeing. I don't think your conclusion is far off.
Appreciate the detail. I ended up in about the same place using SGLang.
Re: what are you all running this week on dual 3090 - thetagrid15 - 09-08-2026
Appreciate the detail. That matches what I saw on 4090. I got better results once I stopped chasing one metric and just looked at whether the workflow felt usable.
Re: what are you all running this week on dual 3090 - quiettoken - 09-08-2026
Good point. I ended up in about the same place using llama.cpp. I got better results once I stopped chasing one metric and just looked at whether the workflow felt usable.