RAG chunking that actually helped with TensorRT-LLM

[thetagrid15]

same here on my side.

[rocmcrate26]

same here on my side.

Maybe, but I had the opposite result. I wouldn't optimize around that first. On my side the bigger swing came from the runtime and not the model. Worth testing the simpler explanation before adopting the more interesting one.

[localtinker]

That matches what I was seeing. I ended up in about the same place using vLLM. The biggest change for me was simplifying the stack instead of adding one more optimization layer.