few-shot when you actually need it on B200

[rasterhall]

This keeps coming up in conversation so I figured I'd start a thread.

The more I work with local setups, the more it seems like KV cache pressure decides everything downstream.

Curious whether that lines up with what other people are seeing.

My use case is pretty ordinary: local chat, some document work, and occasional batch jobs.

[thetafield]

This keeps coming up in conversation so I figured I'd start a thread.

same here on my side.

[nanopatch]

subscribed, curious where this lands.