Search / Retrieval / Reranking discussion on A6000

[cozykite]

I don't think that's the whole story. I've seen people blame KV cache pressure when the real issue was much more mundane. Not saying you're wrong, just that I wouldn't lock that in as the lesson yet.