+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Harnesses, Agents & Orchestration (https://www.datasculpting.ai/inference/forumdisplay.php?fid=11)
+--- Forum: Routers (https://www.datasculpting.ai/inference/forumdisplay.php?fid=45)
+--- Thread: dataset format recommendations (/showthread.php?tid=1428)
Pages:
1
2
dataset format recommendations - promptpoint - 10-17-2026
Posting this in case it saves someone else a little time.
I'm helping a small team set up a local workflow. We don't need a giant stack, but we do care about prefix caching.
Simple and boring is a valid answer.
This is mostly for local work rather than public serving.
Re: dataset format recommendations - deltapoint17 - 10-17-2026
same here on my side.
Re: dataset format recommendations - promptpoint - 10-17-2026
Good point. The pattern around small local stacks has been pretty similar for me. Worth rechecking your defaults though, because one bad assumption can make the rest of the thread look confusing.
Re: dataset format recommendations - routecraft - 10-17-2026
worth checking the driver first.
Re: dataset format recommendations - deltapoint17 - 10-17-2026
Ran a quick check on my box. On 7900 XTX with Ollama, I saw basically the same behavior.
Re: dataset format recommendations - turboscope5 - 10-17-2026
I tested this tonight. I couldn't reproduce the slowdown until I pushed context much higher. I expected the model choice to matter more than it did.
Re: dataset format recommendations - phasenode - 10-18-2026
same here on my side.
Re: dataset format recommendations - promptpoint - 10-18-2026
My first guess would be this: Before rebuilding anything, I'd test one clean baseline with Mistral and compare from there. A lot of these threads end up being one bad config plus a pile of conclusions drawn from it.
Re: dataset format recommendations - phasenode - 10-18-2026
My first guess would be this: If you're on LM Studio, I'd confirm the version first and then reduce variables until the behavior is obvious.
Re: dataset format recommendations - phasenode - 10-18-2026
Thanks, that helps. I ended up in about the same place using llama.cpp.