+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Inference Engines (https://www.datasculpting.ai/inference/forumdisplay.php?fid=9)
+--- Forum: Other Engines (https://www.datasculpting.ai/inference/forumdisplay.php?fid=39)
+--- Thread: prefill bottlenecks on longer prompts: early impressions (/showthread.php?tid=34)
prefill bottlenecks on longer prompts: early impressions - dockkite69 - 08-25-2026
Posting this in case it saves someone else a little time.
I'm not trying to build a perfect architecture diagram. I just want a sane starting point around vLLM and 4090.
Simple and boring is a valid answer.
I care more about stability than squeezing the last possible tokens/sec out of it.
Re: prefill bottlenecks on longer prompts: early impressions - deltagrid - 08-25-2026
A couple questions before I guess too hard. What model, what runtime, and roughly what context size?