10-01-2026, 09:19 PM
Wanted to compare notes with people doing similar things.
I'm mostly working with Phi in Ollama on A6000. The part I'm trying to understand is continuous batching.
Not looking for the perfect answer, mostly what worked in real use.
I'm mostly working with Phi in Ollama on A6000. The part I'm trying to understand is continuous batching.
Not looking for the perfect answer, mostly what worked in real use.
small models can still be useful