+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Harnesses, Agents & Orchestration (https://www.datasculpting.ai/inference/forumdisplay.php?fid=11)
+--- Forum: Workflows / Automation (https://www.datasculpting.ai/inference/forumdisplay.php?fid=47)
+--- Thread: frontier API value lately with Ollama (/showthread.php?tid=599)
Pages:
1
2
frontier API value lately with Ollama - routecraft - 09-29-2026
Spent some time on this today and wanted to compare notes.
I've gone back and forth between SGLang and SGLang for Kimi. The interesting difference for me has been HBM constraints.
What would you pick if you had to live with it for a few months?
For context, I'm not chasing leaderboard numbers. I mostly want the system to feel consistent day to day.
Re: frontier API value lately with Ollama - batchsize - 09-29-2026
I'd approach it in a boring order. Before rebuilding anything, I'd test one clean baseline with DeepSeek and compare from there. Once you know the plain path works, then add the faster path back in piece by piece.
I'm happy to test one more variant if you want a second data point.
Re: frontier API value lately with Ollama - routecraft - 09-30-2026
I'd approach it in a boring order. Start by verifying the driver/runtime combo, then test with a smaller prompt, then look at structured output. Once you know the plain path works, then add the faster path back in piece by piece.
Re: frontier API value lately with Ollama - routecraft - 10-01-2026
Thanks, that helps. That matches what I saw on dual 3090. Worth rechecking your defaults though, because one bad assumption can make the rest of the thread look confusing.
Re: frontier API value lately with Ollama - dockworks23 - 10-01-2026
Quote:Thanks, that helps. That matches what I saw on dual 3090. Worth rechecking your defaults though, because one bad assumption can make the rest of the t
A couple things I'd check first. Start by verifying the driver/runtime combo, then test with a smaller prompt, then look at prefix caching. A lot of these threads end up being one bad config plus a pile of conclusions drawn from it.
Re: frontier API value lately with Ollama - routecraft - 10-01-2026
I think the bottleneck is somewhere else. I've seen people blame NVFP4 when the real issue was much more mundane. Not saying you're wrong, just that I wouldn't lock that in as the lesson yet.
Re: frontier API value lately with Ollama - sparrowbrook - 10-03-2026
My first guess would be this: On 4090, the easiest win is often to back off the aggressive settings and see whether the problem survives. A lot of these threads end up being one bad config plus a pile of conclusions drawn from it.
Re: frontier API value lately with Ollama - routecraft - 10-04-2026
Appreciate the detail. That matches what I saw on A6000. Worth rechecking your defaults though, because one bad assumption can make the rest of the thread look confusing.
Re: frontier API value lately with Ollama - batchember - 10-04-2026
subscribed, curious where this lands.
Re: frontier API value lately with Ollama - routecraft - 10-05-2026
A couple questions before I guess too hard. Are you testing this on A6000 or something else?