+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: RAG & Knowledge Systems (https://www.datasculpting.ai/inference/forumdisplay.php?fid=14)
+--- Forum: RAG (https://www.datasculpting.ai/inference/forumdisplay.php?fid=58)
+--- Thread: hybrid retrieval results on GB200 (/showthread.php?tid=1258)
hybrid retrieval results on GB200 - bytesupply76 - 10-16-2026
Did a bit of testing and thought I'd post the result here.
I tried this with Kimi using TensorRT-LLM on mini PC. The headline result was better than I expected, but structured output still ended up being the limiting factor.
Happy to post more details if anyone wants them.
Re: hybrid retrieval results on GB200 - cobaltworks25 - 10-16-2026
Quote:Did a bit of testing and thought I'd post the result here.
A couple questions before I guess too hard. Is this local-only or are you routing anything externally? Turns out 'just one more test' was a lie.
Re: hybrid retrieval results on GB200 - tinycontext - 10-16-2026
A couple things I'd check first. If you're on Ollama, I'd confirm the version first and then reduce variables until the behavior is obvious. Once you know the plain path works, then add the faster path back in piece by piece.
I'm happy to test one more variant if you want a second data point.