context size vs speed in GGUF with vLLM

[phasescope]

Good point. The pattern around reranking has been pretty similar for me. Worth rechecking your defaults though, because one bad assumption can make th

My first guess would be this: Start by verifying the driver/runtime combo, then test with a smaller prompt, then look at KV cache pressure.