comparing models without fooling yourself for Phi

[batchfield55]

My first guess would be this: Start by verifying the driver/runtime combo, then test with a smaller prompt, then look at speculative decoding. Once yo

I think the bottleneck is somewhere else. I get why that seems right, but on dual 3090 the bottleneck felt elsewhere. Could be a workload difference, but I wouldn't generalize too fast from one setup.

There are a few different conversations hiding inside this thread, so being explicit about the goal helps.