Used A6000 worth it now

[sanevault]

Spent some time on this today and wanted to compare notes.

I'm comparing two approaches here: keep it simple on Threadripper box, or squeeze more out of the stack and deal with the complexity.

Interested in tradeoffs more than fan loyalty.

[vrambandit]

My first guess would be this: Start by verifying the driver/runtime combo, then test with a smaller prompt, then look at tool calling. Not glamorous advice, but repeatable baselines save a lot of time.