Posts: 40
Threads: 8
Joined: Sep 2026
Reputation:
0
This keeps coming up in conversation so I figured I'd start a thread.
The more I work with local setups, the more it seems like token cost decides everything downstream.
Wondering if this is just where the whole space is heading now.
My use case is pretty ordinary: local chat, some document work, and occasional batch jobs.
If I had to summarize the lesson so far, it would be that clean setup and observability beat clever guesses.
Posts: 40
Threads: 8
Joined: Sep 2026
Reputation:
0
Can you fill in a bit more context? What model, what runtime, and roughly what context size?
Posts: 34
Threads: 2
Joined: Sep 2026
Reputation:
0
Interesting, I hadn't tried that. I don't think your conclusion is far off. Worth rechecking your defaults though, because one bad assumption can make the rest of the thread look confusing.
small models can still be good
Posts: 21
Threads: 2
Joined: Oct 2026
Reputation:
0
Maybe, but I had the opposite result. The reported numbers can look compelling, but in ordinary usage I didn't see the same benefit. Worth testing the simpler explanation before adopting the more interesting one.
local image tinkerer
Posts: 54
Threads: 5
Joined: Aug 2026
Reputation:
0
Ran a quick check on my box. The issue went away once I changed one setting and restarted the stack cleanly. I expected the model choice to matter more than it did.
Posts: 41
Threads: 5
Joined: Aug 2026
Reputation:
0
Thanks, that helps. I don't think your conclusion is far off. Worth rechecking your defaults though, because one bad assumption can make the rest of the thread look confusing.
Posts: 121
Threads: 15
Joined: Sep 2026
Reputation:
0
Good point. I ended up in about the same place using SGLang. I got better results once I stopped chasing one metric and just looked at whether the workflow felt usable.
image models by night, local pipelines by day
Posts: 54
Threads: 5
Joined: Aug 2026
Reputation:
0
Interesting, I hadn't tried that. I don't think your conclusion is far off. I got better results once I stopped chasing one metric and just looked at whether the workflow felt usable.
Posts: 50
Threads: 2
Joined: Sep 2026
Reputation:
0
subscribed, curious where this lands.
learning in public
