+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: RAG & Knowledge Systems (https://www.datasculpting.ai/inference/forumdisplay.php?fid=14)
+--- Forum: RAG (https://www.datasculpting.ai/inference/forumdisplay.php?fid=58)
+--- Thread: search and retrieval tuning tips with SGLang (/showthread.php?tid=880)
search and retrieval tuning tips with SGLang - vectorhouse71 - 10-07-2026
Posting this in case it saves someone else a little time.
I'm helping a small team set up a local workflow. We don't need a giant stack, but we do care about VRAM headroom.
Simple and boring is a valid answer.
For context, I'm not chasing leaderboard numbers. I mostly want the system to feel consistent day to day.
Re: search and retrieval tuning tips with SGLang - homelabhenry - 10-07-2026
I tested this tonight. The issue went away once I changed one setting and restarted the stack cleanly. I expected the model choice to matter more than it did.
Re: search and retrieval tuning tips with SGLang - modbench - 10-08-2026
My first guess would be this: Before rebuilding anything, I'd test one clean baseline with Mistral and compare from there. Not glamorous advice, but repeatable baselines save a lot of time.
Re: search and retrieval tuning tips with SGLang - orbitgarden - 10-08-2026
subscribed, curious where this lands.
Re: search and retrieval tuning tips with SGLang - homelabhenry - 10-09-2026
Quote:subscribed, curious where this lands.
Can you fill in a bit more context? Do you see the same thing with a smaller batch or shorter prompt?
Re: search and retrieval tuning tips with SGLang - batchshed - 10-09-2026
following this because I'm in the same boat.
Re: search and retrieval tuning tips with SGLang - modrunner27 - 10-09-2026
subscribed, curious where this lands.
If you post your exact settings, people here can probably narrow it down pretty fast.