power delivery for 24/7 serving

[homelabhenry]

subscribed, curious where this lands.

[plainscope50]

subscribed, curious where this lands.

A couple questions before I guess too hard. What model, what runtime, and roughly what context size?

[homelabhenry]

What worked for me was pretty simple. Before rebuilding anything, I'd test one clean baseline with Mixtral and compare from there. Not glamorous advice, but repeatable baselines save a lot of time.

[phasenode]

Need a little more detail here. What model, what runtime, and roughly what context size?

[batchwire]

I hit almost the same issue last night.