vLLM vs SGLang for small teams on M3 Max

[deltapoint17]

This keeps coming up in conversation so I figured I'd start a thread.

It feels like the conversation has shifted hard toward inference and serving details lately, and honestly that makes sense once you start dealing with workstation thermals.

Wondering if this is just where the whole space is heading now.

[turboyard94]

This keeps coming up in conversation so I figured I'd start a thread.

that fixed it for me too.

[audioforge]

I'd approach it in a boring order. Start by verifying the driver/runtime combo, then test with a smaller prompt, then look at continuous batching. A lot of these threads end up being one bad config plus a pile of conclusions drawn from it.

I'm happy to test one more variant if you want a second data point.