hybrid retrieval results: early impressions

[tinycontext]

subscribed, curious where this lands.

There are a few different conversations hiding inside this thread, so being explicit about the goal helps.

[homelabhenry]

Thanks, that helps. I don't think your conclusion is far off. The biggest change for me was simplifying the stack instead of adding one more optimization layer.

[tinycontext]

I hit almost the same issue last night.

[routecrate]

Did a small comparison this afternoon. On Threadripper box with vLLM, I saw basically the same behavior.

[modbench]

Did a small comparison this afternoon. The issue went away once I changed one setting and restarted the stack cleanly. The surprise was that memory pressure showed up before throughput collapsed.