09-04-2026, 02:14 AM
I'm trying to sanity-check my setup.
I'm mostly working with Llama in SGLang on 4090. The part I'm trying to understand is prefill/decode split.
Not looking for the perfect answer, mostly what worked in real use.
This is mostly for local work rather than public serving.
I'm mostly working with Llama in SGLang on 4090. The part I'm trying to understand is prefill/decode split.
Not looking for the perfect answer, mostly what worked in real use.
This is mostly for local work rather than public serving.