Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
latency vs throughput for small deployments on dual 3090
#4
Quote:Did a small comparison this afternoon. The issue went away once I changed one setting and restarted the stack cleanly. The main thing that moved was l

I'd start with the simpler stack.

There are a few different conversations hiding inside this thread, so being explicit about the goal helps.
small models can still be good
Reply


Messages In This Thread
Re: latency vs throughput for small deployments on dual 3090 - by kernelrunner - 10-14-2026, 04:20 AM

Forum Jump:


Users browsing this thread: 1 Guest(s)