system prompt habits that helped: what are you seeing?

[northgarden37]

I hit almost the same issue last night.

[promptspark58]

that fixed it for me too.

[mlxgarden]

that fixed it for me too.

I'd approach it in a boring order. Before rebuilding anything, I'd test one clean baseline with Llama and compare from there. At this point the real bottleneck is my patience.

[thetafield]

Maybe, but I had the opposite result. I get why that seems right, but on A6000 the bottleneck felt elsewhere. Not saying you're wrong, just that I wouldn't lock that in as the lesson yet.

[northgarden37]

Thanks, that helps. I don't think your conclusion is far off. The biggest change for me was simplifying the stack instead of adding one more optimization layer.

[thetaarch43]

My first guess would be this: Start by verifying the driver/runtime combo, then test with a smaller prompt, then look at long context.