Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
disaggregated inference thoughts: what are you seeing?
#1
Posting this in case it saves someone else a little time.

I'm helping a small team set up a local workflow. We don't need a giant stack, but we do care about tokens/sec.

Would rather learn from your scars than collect my own all at once.

For context, I'm not chasing leaderboard numbers. I mostly want the system to feel consistent day to day.

The part that surprised me was how much the boring details mattered more than the model choice.
Reply


Messages In This Thread
disaggregated inference thoughts: what are you seeing? - by localpatch78 - 10-11-2026, 02:41 PM

Forum Jump:


Users browsing this thread: 1 Guest(s)