+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: RAG & Knowledge Systems (https://www.datasculpting.ai/inference/forumdisplay.php?fid=14)
+--- Forum: RAG (https://www.datasculpting.ai/inference/forumdisplay.php?fid=58)
+--- Thread: hybrid retrieval results for Mixtral (/showthread.php?tid=1032)
Pages:
1
2
hybrid retrieval results for Mixtral - bytelab - 10-10-2026
Thought I'd post a small recipe because a lot of threads ask similar things.
Just sharing a setup that ended up working reasonably well: Mistral, llama.cpp, M3 Max, with a lot of attention on tokens/sec.
If someone wants the exact settings I can clean them up and post them.
My use case is pretty ordinary: local chat, some document work, and occasional batch jobs.
Re: hybrid retrieval results for Mixtral - homelabhenry - 10-11-2026
What worked for me was pretty simple. Before rebuilding anything, I'd test one clean baseline with Kimi and compare from there. Once you know the plain path works, then add the faster path back in piece by piece.
Re: hybrid retrieval results for Mixtral - mintstream9 - 10-11-2026
My first guess would be this: On 5090, the easiest win is often to back off the aggressive settings and see whether the problem survives. A lot of these threads end up being one bad config plus a pile of conclusions drawn from it.
Re: hybrid retrieval results for Mixtral - batchsize - 10-11-2026
Maybe, but I had the opposite result. The reported numbers can look compelling, but in ordinary usage I didn't see the same benefit. Not saying you're wrong, just that I wouldn't lock that in as the lesson yet.
I'm happy to test one more variant if you want a second data point.
Re: hybrid retrieval results for Mixtral - cacheworks - 10-12-2026
Not sure I agree there. I get why that seems right, but on dual 3090 the bottleneck felt elsewhere. Not saying you're wrong, just that I wouldn't lock that in as the lesson yet.
Re: hybrid retrieval results for Mixtral - kernelghost - 10-12-2026
What exact setup are you running? Are you testing this on Threadripper box or something else?
Re: hybrid retrieval results for Mixtral - tinycontext - 10-12-2026
A couple questions before I guess too hard. Are you testing this on B200 or something else?
Re: hybrid retrieval results for Mixtral - labpath - 10-12-2026
Interesting, I hadn't tried that. I don't think your conclusion is far off.
Re: hybrid retrieval results for Mixtral - homelabhenry - 10-14-2026
Quote:Interesting, I hadn't tried that. I don't think your conclusion is far off.
A couple questions before I guess too hard. Are you testing this on 4090 or something else?
Re: hybrid retrieval results for Mixtral - relayforge - 10-14-2026
Quote:Quote:Interesting, I hadn't tried that. I don't think your conclusion is far off.
subscribed, curious where this lands.