+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: RAG & Knowledge Systems (https://www.datasculpting.ai/inference/forumdisplay.php?fid=14)
+--- Forum: Search / Retrieval / Reranking (https://www.datasculpting.ai/inference/forumdisplay.php?fid=61)
+--- Thread: Search / Retrieval / Reranking discussion for Gemma (/showthread.php?tid=1203)
Search / Retrieval / Reranking discussion for Gemma - tensorfield88 - 10-15-2026
Posting this in case it saves someone else a little time.
If you were starting from scratch today for this use case, what would you choose first and what would you ignore?
Simple and boring is a valid answer.
Re: Search / Retrieval / Reranking discussion for Gemma - cobaltlab - 10-17-2026
What worked for me was pretty simple. Start by verifying the driver/runtime combo, then test with a smaller prompt, then look at speculative decoding.
Re: Search / Retrieval / Reranking discussion for Gemma - deltagrid - 10-17-2026
I'd start with the simpler stack.
Re: Search / Retrieval / Reranking discussion for Gemma - llmarch51 - 10-17-2026
A couple things I'd check first. Start by verifying the driver/runtime combo, then test with a smaller prompt, then look at KV cache pressure.
Re: Search / Retrieval / Reranking discussion for Gemma - localcraft17 - 10-17-2026
My first guess would be this: Before rebuilding anything, I'd test one clean baseline with Phi and compare from there.
Re: Search / Retrieval / Reranking discussion for Gemma - relaysupply90 - 10-17-2026
Appreciate the detail. I don't think your conclusion is far off. Worth rechecking your defaults though, because one bad assumption can make the rest of the thread look confusing.
Re: Search / Retrieval / Reranking discussion for Gemma - vectorcrate98 - 10-17-2026
My first guess would be this: On B200, the easiest win is often to back off the aggressive settings and see whether the problem survives.
Re: Search / Retrieval / Reranking discussion for Gemma - circuitwire58 - 10-18-2026
I'd approach it in a boring order. Before rebuilding anything, I'd test one clean baseline with Kimi and compare from there. A lot of these threads end up being one bad config plus a pile of conclusions drawn from it.
Re: Search / Retrieval / Reranking discussion for Gemma - deepseeked - 10-18-2026
I'd approach it in a boring order. Before rebuilding anything, I'd test one clean baseline with Gemma and compare from there.
Re: Search / Retrieval / Reranking discussion for Gemma - turbosupply83 - 10-18-2026
I don't think that's the whole story. I wouldn't optimize around that first. On my side the bigger swing came from the runtime and not the model. Not saying you're wrong, just that I wouldn't lock that in as the lesson yet.