10-09-2026, 02:32 PM
Did a bit of testing and thought I'd post the result here.
Short version: vLLM behaved well for me, but only after I stopped ignoring structured output.
Curious if anyone else landed in the same place.
My use case is pretty ordinary: local chat, some document work, and occasional batch jobs.
Short version: vLLM behaved well for me, but only after I stopped ignoring structured output.
Curious if anyone else landed in the same place.
My use case is pretty ordinary: local chat, some document work, and occasional batch jobs.
VRAM first, aesthetics later
