+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: RAG & Knowledge Systems (https://www.datasculpting.ai/inference/forumdisplay.php?fid=14)
+--- Forum: Search / Retrieval / Reranking (https://www.datasculpting.ai/inference/forumdisplay.php?fid=61)
+--- Thread: Search / Retrieval / Reranking discussion with SGLang (/showthread.php?tid=523)
Search / Retrieval / Reranking discussion with SGLang - llmtrack - 09-28-2026
Spent some time on this today and wanted to compare notes.
I'm comparing two approaches here: keep it simple on mini PC, or squeeze more out of the stack and deal with the complexity.
What would you pick if you had to live with it for a few months?
My use case is pretty ordinary: local chat, some document work, and occasional batch jobs.
Re: Search / Retrieval / Reranking discussion with SGLang - fusedhall - 09-30-2026
Quote:Spent some time on this today and wanted to compare notes.
A couple things I'd check first. Before rebuilding anything, I'd test one clean baseline with Qwen and compare from there. Not glamorous advice, but repeatable baselines save a lot of time.
Re: Search / Retrieval / Reranking discussion with SGLang - turboscope - 09-30-2026
I'd approach it in a boring order. Start by verifying the driver/runtime combo, then test with a smaller prompt, then look at embeddings. Not glamorous advice, but repeatable baselines save a lot of time.