Inference › LLM Models › Proprietary / Frontier v
70B still worth the pain

Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Linear Mode
70B still worth the pain
batchsize Offline
Member
***
Posts: 137
Threads: 16
Joined: Sep 2026
Reputation: 0
#8
10-19-2026, 02:09 AM
That matches what I was seeing. That matches what I saw on A6000. I got better results once I stopped chasing one metric and just looked at whether the workflow felt usable.
throughput matters, but tail latency matters too
Find
Reply


Messages In This Thread
70B still worth the pain - by turboscope - 10-19-2026, 12:02 AM
Re: 70B still worth the pain - by fastbench - 10-18-2026, 05:05 PM
Re: 70B still worth the pain - by routecrate - 10-18-2026, 09:38 PM
Re: 70B still worth the pain - by fastscope - 10-18-2026, 10:51 PM
Re: 70B still worth the pain - by ggufwire22 - 10-19-2026, 01:03 AM
Re: 70B still worth the pain - by mlxgarden - 10-19-2026, 01:32 AM
Re: 70B still worth the pain - by turboscope5 - 10-19-2026, 02:33 AM
Re: 70B still worth the pain - by batchsize - 10-19-2026, 02:09 AM
Re: 70B still worth the pain - by quietsupply11 - 10-19-2026, 03:10 AM
Re: 70B still worth the pain - by vramghost - 10-19-2026, 03:35 AM
Re: 70B still worth the pain - by tinycontext - 10-19-2026, 05:51 AM
Re: 70B still worth the pain - by amberkite - 10-18-2026, 09:16 PM

Forum Jump:


Users browsing this thread: 1 Guest(s)