+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: RAG & Knowledge Systems (https://www.datasculpting.ai/inference/forumdisplay.php?fid=14)
+--- Forum: Search / Retrieval / Reranking (https://www.datasculpting.ai/inference/forumdisplay.php?fid=61)
+--- Thread: Search / Retrieval / Reranking discussion (/showthread.php?tid=683)
Pages:
1
2
Re: Search / Retrieval / Reranking discussion - mellowwire98 - 10-06-2026
Can you fill in a bit more context? Is this local-only or are you routing anything externally?
Re: Search / Retrieval / Reranking discussion - sparrowworks88 - 10-07-2026
Need a little more detail here. What model, what runtime, and roughly what context size?
Re: Search / Retrieval / Reranking discussion - cudacrate - 10-07-2026
Maybe, but I had the opposite result. I've seen people blame batch size when the real issue was much more mundane. Worth testing the simpler explanation before adopting the more interesting one.
Re: Search / Retrieval / Reranking discussion - tensorport - 10-07-2026
Quote:Maybe, but I had the opposite result. I've seen people blame batch size when the real issue was much more mundane. Worth testing the simpler explanati
Appreciate the detail. I don't think your conclusion is far off.
Re: Search / Retrieval / Reranking discussion - southtinker - 10-07-2026
Ran a quick check on my box. My result was mixed: first run looked bad, second run looked normal. The surprise was that memory pressure showed up before throughput collapsed.