+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Inference Engines (https://www.datasculpting.ai/inference/forumdisplay.php?fid=9)
+--- Forum: Other Engines (https://www.datasculpting.ai/inference/forumdisplay.php?fid=39)
+--- Thread: prefill bottlenecks on longer prompts on A6000 (/showthread.php?tid=466)
Pages:
1
2
Re: prefill bottlenecks on longer prompts on A6000 - mintstream9 - 10-03-2026
Appreciate the detail. The pattern around MoE serving has been pretty similar for me. I got better results once I stopped chasing one metric and just looked at whether the workflow felt usable.
Re: prefill bottlenecks on longer prompts on A6000 - mlxgarden - 10-04-2026
I don't think that's the whole story. I get why that seems right, but on dual 3090 the bottleneck felt elsewhere.
Re: prefill bottlenecks on longer prompts on A6000 - tinycontext - 10-05-2026
Quote:I don't think that's the whole story. I get why that seems right, but on dual 3090 the bottleneck felt elsewhere.
Ran a quick check on my box. My result was mixed: first run looked bad, second run looked normal. I expected the model choice to matter more than it did.
I'm happy to test one more variant if you want a second data point. Turns out 'just one more test' was a lie.