Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
which hosted model is actually worth paying for
#1
I figured I'd ask here before changing everything.

I'm trying to decide whether to keep the current stack or change course. Main question is how people here are handling tokens/sec.

Would appreciate practical advice rather than theory.

This is mostly for local work rather than public serving.
routing beats arguing over one perfect model
Reply
#2
I tested this tonight. On dual 3090 with Ollama, I saw basically the same behavior. The main thing that moved was latency, not quality.
routing beats arguing over one perfect model
Reply


Forum Jump:


Users browsing this thread: 1 Guest(s)