Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
cleaning small domain dataset with llama.cpp
#1
I'm trying to sanity-check my setup.

I'm mostly working with Mistral in Ollama on 5090. The part I'm trying to understand is HBM constraints.

If you've tested something similar, what did you keep and what did you stop bothering with?

I expected the biggest difference to come from the model, but the stack around it mattered at least as much.
Reply
#2
Did a small comparison this afternoon. On 7900 XTX with LM Studio, I saw basically the same behavior. The surprise was that memory pressure showed up before throughput collapsed.
routing beats arguing over one perfect model
Reply


Forum Jump:


Users browsing this thread: 1 Guest(s)