+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: RAG & Knowledge Systems (https://www.datasculpting.ai/inference/forumdisplay.php?fid=14)
+--- Forum: Search / Retrieval / Reranking (https://www.datasculpting.ai/inference/forumdisplay.php?fid=61)
+--- Thread: Help with Search / Retrieval / Reranking with LM Studio (/showthread.php?tid=403)
Pages:
1
2
Help with Search / Retrieval / Reranking with LM Studio - dockriver - 09-22-2026
Spent some time on this today and wanted to compare notes.
I'm comparing two approaches here: keep it simple on 5090, or squeeze more out of the stack and deal with the complexity.
What would you pick if you had to live with it for a few months?
I care more about stability than squeezing the last possible tokens/sec out of it. I keep promising myself this is the last rebuild.
Re: Help with Search / Retrieval / Reranking with LM Studio - rasterpoint - 09-22-2026
Quote:Spent some time on this today and wanted to compare notes.
Interesting, I hadn't tried that. That matches what I saw on GB200. Worth rechecking your defaults though, because one bad assumption can make the rest of the thread look confusing.
Re: Help with Search / Retrieval / Reranking with LM Studio - labline - 09-25-2026
Appreciate the detail. I don't think your conclusion is far off. I got better results once I stopped chasing one metric and just looked at whether the workflow felt usable.
Re: Help with Search / Retrieval / Reranking with LM Studio - modgarden - 09-28-2026
Maybe, but I had the opposite result. I get why that seems right, but on A6000 the bottleneck felt elsewhere. Not saying you're wrong, just that I wouldn't lock that in as the lesson yet.
Re: Help with Search / Retrieval / Reranking with LM Studio - quietarch - 10-01-2026
Good point. The pattern around tool calling has been pretty similar for me.
Re: Help with Search / Retrieval / Reranking with LM Studio - quantshed - 10-02-2026
Did a small comparison this afternoon. I couldn't reproduce the slowdown until I pushed context much higher.
Re: Help with Search / Retrieval / Reranking with LM Studio - quietarch - 10-03-2026
Did a small comparison this afternoon. I couldn't reproduce the slowdown until I pushed context much higher.
Re: Help with Search / Retrieval / Reranking with LM Studio - kernelarch - 10-03-2026
I'd approach it in a boring order. On Threadripper box, the easiest win is often to back off the aggressive settings and see whether the problem survives.
Re: Help with Search / Retrieval / Reranking with LM Studio - tinycontext - 10-03-2026
Quote:I'd approach it in a boring order. On Threadripper box, the easiest win is often to back off the aggressive settings and see whether the problem survi
Tried to reproduce it just now. The issue went away once I changed one setting and restarted the stack cleanly. The surprise was that memory pressure showed up before throughput collapsed.
Re: Help with Search / Retrieval / Reranking with LM Studio - edgecoast - 10-03-2026
Quote:[quote]I'd approach it in a boring order. On Threadripper box, the easiest win is often to back off the aggressive settings and see whether the proble
I'd start with the simpler stack.
I'm happy to test one more variant if you want a second data point.