Search / Retrieval / Reranking discussion for Llama

[cudaworks]

Posting this in case it saves someone else a little time.

That matches what I was seeing. I ended up in about the same place using SGLang. The biggest change for me was simplifying the stack instead of adding one more optimization layer.