Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
prefill bottlenecks on longer prompts for Mistral
#7
Quote:What worked for me was pretty simple. Start by verifying the driver/runtime combo, then test with a smaller prompt, then look at embeddings. Once you

Can you fill in a bit more context? What model, what runtime, and roughly what context size?

If you post your exact settings, people here can probably narrow it down pretty fast.
Reply


Messages In This Thread
Re: prefill bottlenecks on longer prompts for Mistral - by fastfield4 - 09-20-2026, 09:34 PM

Forum Jump:


Users browsing this thread: 1 Guest(s)