Posts: 37
Threads: 5
Joined: Oct 2026
Reputation:
0
Spent some time on this today and wanted to compare notes.
I'm comparing two approaches here: keep it simple on mini PC, or squeeze more out of the stack and deal with the complexity.
What would you pick if you had to live with it for a few months?
My use case is pretty ordinary: local chat, some document work, and occasional batch jobs.
Posts: 50
Threads: 4
Joined: Sep 2026
Reputation:
0
worth checking the driver first.
Posts: 45
Threads: 3
Joined: Sep 2026
Reputation:
0
A couple questions before I guess too hard. What model, what runtime, and roughly what context size?
Posts: 33
Threads: 4
Joined: Aug 2026
Reputation:
0
I think the bottleneck is somewhere else. I wouldn't optimize around that first. On my side the bigger swing came from the runtime and not the model. Worth testing the simpler explanation before adopting the more interesting one.
Posts: 51
Threads: 8
Joined: Sep 2026
Reputation:
0
Not sure I agree there. I wouldn't optimize around that first. On my side the bigger swing came from the runtime and not the model.
learning in public
Posts: 45
Threads: 8
Joined: Sep 2026
Reputation:
0
Not sure I agree there. I get why that seems right, but on 5090 the bottleneck felt elsewhere. Worth testing the simpler explanation before adopting the more interesting one.
team red, team patient
Posts: 50
Threads: 4
Joined: Sep 2026
Reputation:
0
I'd approach it in a boring order. If you're on vLLM, I'd confirm the version first and then reduce variables until the behavior is obvious.
Posts: 51
Threads: 8
Joined: Sep 2026
Reputation:
0
I don't think that's the whole story. The reported numbers can look compelling, but in ordinary usage I didn't see the same benefit.
learning in public
