Search / Retrieval / Reranking discussion for Gemma

[dockworks23]

Did a bit of testing and thought I'd post the result here.

After a few runs on mini PC, I ended up with decent performance, but the setup only really made sense once I adjusted around HBM constraints.

Nothing scientific here, just practical notes from a real setup.

For context, I'm not chasing leaderboard numbers. I mostly want the system to feel consistent day to day.

[deepseeked]

I hit almost the same issue last night.

[localcraft17]

Can you fill in a bit more context? What model, what runtime, and roughly what context size?