Help with Embeddings

[rasterhall]

Spent some time on this today and wanted to compare notes.

I'm comparing two approaches here: keep it simple on A6000, or squeeze more out of the stack and deal with the complexity.

Trying to avoid rebuilding the whole thing twice.

I care more about stability than squeezing the last possible tokens/sec out of it.

[paperport64]

Can you fill in a bit more context? Do you see the same thing with a smaller batch or shorter prompt? Turns out 'just one more test' was a lie.

[rasterhall]

I'd start with the simpler stack.