+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: LLM Models (https://www.datasculpting.ai/inference/forumdisplay.php?fid=12)
+--- Forum: Open Weight (https://www.datasculpting.ai/inference/forumdisplay.php?fid=49)
+---- Forum: Local (https://www.datasculpting.ai/inference/forumdisplay.php?fid=51)
+---- Thread: working config for 32k context (/showthread.php?tid=779)
working config for 32k context - rocmcrate26 - 10-05-2026
Spent some time on this today and wanted to compare notes.
I've gone back and forth between SGLang and vLLM for Phi. The interesting difference for me has been structured output.
Interested in tradeoffs more than fan loyalty.
I care more about stability than squeezing the last possible tokens/sec out of it.