+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: RAG & Knowledge Systems (https://www.datasculpting.ai/inference/forumdisplay.php?fid=14)
+--- Forum: RAG (https://www.datasculpting.ai/inference/forumdisplay.php?fid=58)
+--- Thread: hybrid retrieval results (/showthread.php?tid=854)
Pages:
1
2
hybrid retrieval results - sparrowbrook - 10-07-2026
Thought I'd post a small recipe because a lot of threads ask similar things.
This isn't revolutionary, but the combination of MLX and Threadripper box felt surprisingly usable once I tuned around batch size.
Maybe useful as a starting point for newer folks.
I care more about stability than squeezing the last possible tokens/sec out of it.
Re: hybrid retrieval results - turbogrid11 - 10-07-2026
My first guess would be this: If you're on Ollama, I'd confirm the version first and then reduce variables until the behavior is obvious.
Re: hybrid retrieval results - homelabhenry - 10-09-2026
What worked for me was pretty simple. On dual 3090, the easiest win is often to back off the aggressive settings and see whether the problem survives. Once you know the plain path works, then add the faster path back in piece by piece.
Re: hybrid retrieval results - kernelghost - 10-09-2026
Quote:What worked for me was pretty simple. On dual 3090, the easiest win is often to back off the aggressive settings and see whether the problem survives.
A couple things I'd check first. Before rebuilding anything, I'd test one clean baseline with DeepSeek and compare from there. A lot of these threads end up being one bad config plus a pile of conclusions drawn from it.
Re: hybrid retrieval results - llmtrack - 10-11-2026
I think the bottleneck is somewhere else. I've seen people blame quant drift when the real issue was much more mundane. Worth testing the simpler explanation before adopting the more interesting one.
Re: hybrid retrieval results - nanogarden - 10-12-2026
Quote:I think the bottleneck is somewhere else. I've seen people blame quant drift when the real issue was much more mundane. Worth testing the simpler expl
I don't think that's the whole story. I wouldn't optimize around that first. On my side the bigger swing came from the runtime and not the model.
Re: hybrid retrieval results - bytelab - 10-12-2026
Appreciate the detail. That matches what I saw on A6000. Worth rechecking your defaults though, because one bad assumption can make the rest of the thread look confusing.
Re: hybrid retrieval results - homelabhenry - 10-12-2026
That matches what I was seeing. I don't think your conclusion is far off. Worth rechecking your defaults though, because one bad assumption can make the rest of the thread look confusing.
Re: hybrid retrieval results - sanepeak48 - 10-12-2026
Appreciate the detail. I don't think your conclusion is far off. Worth rechecking your defaults though, because one bad assumption can make the rest of the thread look confusing.
Re: hybrid retrieval results - tinycontext - 10-12-2026
What worked for me was pretty simple. Before rebuilding anything, I'd test one clean baseline with DeepSeek and compare from there. A lot of these threads end up being one bad config plus a pile of conclusions drawn from it.