Inference › Inference Engines › vLLM / SGLang v
continuous batching wins and losses with Ollama

Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Linear Mode
continuous batching wins and losses with Ollama
oxidemixer Offline
Member
***
Posts: 50
Threads: 6
Joined: Sep 2026
Reputation: 0
#1
10-01-2026, 05:30 PM
Spent some time on this today and wanted to compare notes.

I've gone back and forth between Ollama and LM Studio for Qwen. The interesting difference for me has been speculative decoding.

Trying to avoid rebuilding the whole thing twice.
Find
Reply


Messages In This Thread
continuous batching wins and losses with Ollama - by oxidemixer - 10-01-2026, 05:30 PM

Forum Jump:


Users browsing this thread: 1 Guest(s)