08-31-2026, 10:22 PM
Quote:[quote]What worked for me was pretty simple. Start by verifying the driver/runtime combo, then test with a smaller prompt, then look at tokens/sec.[/q
I'd approach it in a boring order. If you're on vLLM, I'd confirm the version first and then reduce variables until the behavior is obvious.
learning in public