MCP server discovery mess on B200

[latentcraft]

This keeps coming up in conversation so I figured I'd start a thread.

One thing I keep noticing is that people underestimate KV cache pressure until they actually try to run this stuff for more than one afternoon.

Wondering if this is just where the whole space is heading now.

[thetafield]

A couple things I'd check first. On A6000, the easiest win is often to back off the aggressive settings and see whether the problem survives. Once you know the plain path works, then add the faster path back in piece by piece.

[rocmpath]

Can you fill in a bit more context? What model, what runtime, and roughly what context size?

[vramforge]

Interesting, I hadn't tried that. That matches what I saw on 4090. The biggest change for me was simplifying the stack instead of adding one more optimization layer.

[nanopatch]

I tested this tonight. My result was mixed: first run looked bad, second run looked normal.

[northquill]

I'd start with the simpler stack.

[orbitbench6]

Need a little more detail here. Do you see the same thing with a smaller batch or shorter prompt?

[turbosupply83]

Maybe, but I had the opposite result. The reported numbers can look compelling, but in ordinary usage I didn't see the same benefit. Not saying you're wrong, just that I wouldn't lock that in as the lesson yet. The logs were somehow calmer than I was.

[relaynest82]

Need a little more detail here. What model, what runtime, and roughly what context size?

[fusedhall]

Thanks, that helps. The pattern around embeddings has been pretty similar for me. Worth rechecking your defaults though, because one bad assumption can make the rest of the thread look confusing.