dataset format recommendations

[phasenode]

Spent some time on this today and wanted to compare notes.

I've gone back and forth between PyTorch and Ollama for FLUX. The interesting difference for me has been KV cache pressure.

Interested in tradeoffs more than fan loyalty.

[plainworks]

Interesting, I hadn't tried that. I don't think your conclusion is far off.

[quantghost]

I tested this tonight. I couldn't reproduce the slowdown until I pushed context much higher. I expected the model choice to matter more than it did.

[quiettoken]

following this because I'm in the same boat.

[thetapath]

I'd approach it in a boring order. If you're on llama.cpp, I'd confirm the version first and then reduce variables until the behavior is obvious.

[phasetrack]

Did a small comparison this afternoon. My result was mixed: first run looked bad, second run looked normal. The main thing that moved was latency, not quality.

[plainline]

Can you fill in a bit more context? What model, what runtime, and roughly what context size?

[batchember]

worth checking the driver first.