Vector Databases discussion on GB200

[macronest]

Wanted to compare notes with people doing similar things.

I'm mostly working with Gemma in LM Studio on 5090. The part I'm trying to understand is prefix caching.

If you've tested something similar, what did you keep and what did you stop bothering with?

This is mostly for local work rather than public serving.

[deltagrid]

My first guess would be this: Start by verifying the driver/runtime combo, then test with a smaller prompt, then look at token cost. Not glamorous advice, but repeatable baselines save a lot of time.