09-17-2026, 09:07 PM
Spent some time on this today and wanted to compare notes.
I've gone back and forth between llama.cpp and vLLM for Gemma. The interesting difference for me has been prefill/decode split.
Interested in tradeoffs more than fan loyalty.
I've gone back and forth between llama.cpp and vLLM for Gemma. The interesting difference for me has been prefill/decode split.
Interested in tradeoffs more than fan loyalty.
trying not to rebuild again
