TTS that sounds less fake: setup advice

[bytegarden73]

This keeps coming up in conversation so I figured I'd start a thread.

The more I work with local setups, the more it seems like prefill/decode split decides everything downstream.

Maybe obvious to some of you, but it clicked for me this week.

For context, I'm not chasing leaderboard numbers. I mostly want the system to feel consistent day to day.

[macropoint]

Appreciate the detail. That matches what I saw on mini PC.

[bytegarden73]

A couple things I'd check first. Start by verifying the driver/runtime combo, then test with a smaller prompt, then look at small local stacks. Once you know the plain path works, then add the faster path back in piece by piece.

[visionforge]

Good point. I don't think your conclusion is far off. The biggest change for me was simplifying the stack instead of adding one more optimization layer.

[quiethouse]

Did a small comparison this afternoon. My result was mixed: first run looked bad, second run looked normal. I expected the model choice to matter more than it did.

[deltastation]

Tried to reproduce it just now. On GB200 with TensorRT-LLM, I saw basically the same behavior.

[cobaltspark]

Tried to reproduce it just now. On GB200 with TensorRT-LLM, I saw basically the same behavior.

Ran a quick check on my box. The issue went away once I changed one setting and restarted the stack cleanly. The surprise was that memory pressure showed up before throughput collapsed.

[amberpatch25]

My first guess would be this: Start by verifying the driver/runtime combo, then test with a smaller prompt, then look at token cost. A lot of these threads end up being one bad config plus a pile of conclusions drawn from it. At this point the real bottleneck is my patience.

[llmtrack]

My first guess would be this: Start by verifying the driver/runtime combo, then test with a smaller prompt, then look at token cost. A lot of these th

Good point. That matches what I saw on 7900 XTX. Worth rechecking your defaults though, because one bad assumption can make the rest of the thread look confusing.

[visionforge]

I'd approach it in a boring order. Start by verifying the driver/runtime combo, then test with a smaller prompt, then look at embeddings. A lot of these threads end up being one bad config plus a pile of conclusions drawn from it.