+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Inference Engines (https://www.datasculpting.ai/inference/forumdisplay.php?fid=9)
+--- Forum: Other Engines (https://www.datasculpting.ai/inference/forumdisplay.php?fid=39)
+--- Thread: prefill bottlenecks on longer prompts: what are you seeing? (/showthread.php?tid=740)
Pages:
1
2
Re: prefill bottlenecks on longer prompts: what are you seeing? - amberpatch25 - 10-14-2026
Good point. The pattern around continuous batching has been pretty similar for me. I got better results once I stopped chasing one metric and just looked at whether the workflow felt usable.
Re: prefill bottlenecks on longer prompts: what are you seeing? - sparrowworks88 - 10-16-2026
Can you fill in a bit more context? Is this local-only or are you routing anything externally?