10-11-2026, 10:24 AM
Wanted to compare notes with people doing similar things.
I'm mostly working with Phi in llama.cpp on B200. The part I'm trying to understand is tool calling.
If you've tested something similar, what did you keep and what did you stop bothering with?
If I had to summarize the lesson so far, it would be that clean setup and observability beat clever guesses.
I'm mostly working with Phi in llama.cpp on B200. The part I'm trying to understand is tool calling.
If you've tested something similar, what did you keep and what did you stop bothering with?
If I had to summarize the lesson so far, it would be that clean setup and observability beat clever guesses.
throughput matters, but tail latency matters too