Posts: 23
Threads: 3
Joined: Aug 2026
Reputation:
0
This keeps coming up in conversation so I figured I'd start a thread.
The more I work with local setups, the more it seems like reranking decides everything downstream.
Curious whether that lines up with what other people are seeing.
My use case is pretty ordinary: local chat, some document work, and occasional batch jobs.
VRAM first, aesthetics later
Posts: 55
Threads: 10
Joined: Sep 2026
Reputation:
0
Good point. I don't think your conclusion is far off. I got better results once I stopped chasing one metric and just looked at whether the workflow felt usable.
Posts: 39
Threads: 4
Joined: Aug 2026
Reputation:
0
Quote:[quote]Good point. I don't think your conclusion is far off. I got better results once I stopped chasing one metric and just looked at whether the wor
I'd approach it in a boring order. If you're on PyTorch, I'd confirm the version first and then reduce variables until the behavior is obvious.
small models can still be good
Posts: 35
Threads: 3
Joined: Sep 2026
Reputation:
0
That matches what I was seeing. I ended up in about the same place using SGLang. Worth rechecking your defaults though, because one bad assumption can make the rest of the thread look confusing.
Posts: 48
Threads: 7
Joined: Sep 2026
Reputation:
0
Ran a quick check on my box. The issue went away once I changed one setting and restarted the stack cleanly.
retrieval before vibes
Posts: 49
Threads: 4
Joined: Sep 2026
Reputation:
0
A couple things I'd check first. Before rebuilding anything, I'd test one clean baseline with DeepSeek and compare from there. Not glamorous advice, but repeatable baselines save a lot of time.
Posts: 199
Threads: 27
Joined: Sep 2026
Reputation:
0
Maybe, but I had the opposite result. The reported numbers can look compelling, but in ordinary usage I didn't see the same benefit.
M3 Max • MLX • local-first
