power delivery for 24/7 serving

[homelabhenry]

Has anyone else run into this?

I'm trying to decide whether to keep the current stack or change course. Main question is how people here are handling prefix caching.

Not looking for the perfect answer, mostly what worked in real use.

This is mostly for local work rather than public serving.

[tensorport]

What worked for me was pretty simple. Start by verifying the driver/runtime combo, then test with a smaller prompt, then look at small local stacks.