+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: RAG & Knowledge Systems (https://www.datasculpting.ai/inference/forumdisplay.php?fid=14)
+--- Forum: Search / Retrieval / Reranking (https://www.datasculpting.ai/inference/forumdisplay.php?fid=61)
+--- Thread: Help with Search / Retrieval / Reranking (/showthread.php?tid=531)
Pages:
1
2
Re: Help with Search / Retrieval / Reranking - rocmrunner40 - 10-09-2026
Quote:Quote:I'd start with the simpler stack.
Can you fill in a bit more context? Are you testing this on A6000 or something else?
Re: Help with Search / Retrieval / Reranking - phasespark - 10-09-2026
I think the bottleneck is somewhere else. I wouldn't optimize around that first. On my side the bigger swing came from the runtime and not the model. Not saying you're wrong, just that I wouldn't lock that in as the lesson yet.
Re: Help with Search / Retrieval / Reranking - cozygrid12 - 10-09-2026
Good point. I ended up in about the same place using LM Studio. Worth rechecking your defaults though, because one bad assumption can make the rest of the thread look confusing.
Re: Help with Search / Retrieval / Reranking - phasescope - 10-09-2026
Need a little more detail here. Are you testing this on GB200 or something else?
Re: Help with Search / Retrieval / Reranking - plainworks - 10-09-2026
Good point. I don't think your conclusion is far off.
Re: Help with Search / Retrieval / Reranking - batchcraft - 10-09-2026
same here on my side.