Inference › Evaluation & Testing › Benchmarks v
comparing models without fooling yourself for Phi

Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Threaded Mode
comparing models without fooling yourself for Phi
batchsize Offline
Member
***
Posts: 137
Threads: 16
Joined: Sep 2026
Reputation: 0
#11
10-06-2026, 08:47 AM
that fixed it for me too.
throughput matters, but tail latency matters too
Find
Reply
benchplease Offline
Member
***
Posts: 123
Threads: 18
Joined: Sep 2026
Reputation: 0
#12
10-06-2026, 10:27 AM
Interesting, I hadn't tried that. I ended up in about the same place using vLLM.
post logs or it didn't happen
Find
Reply
griffinloom66 Offline
Junior Member
**
Posts: 30
Threads: 6
Joined: Aug 2026
Reputation: 0
#13
10-06-2026, 11:53 AM
Quote:Interesting, I hadn't tried that. I ended up in about the same place using vLLM.

Need a little more detail here. Is this local-only or are you routing anything externally?
Find
Reply


Forum Jump:


Users browsing this thread: 1 Guest(s)