how much orchestration is too much: what are you seeing?

[cudaline]

Thought I'd post a small recipe because a lot of threads ask similar things.

This isn't revolutionary, but the combination of TensorRT-LLM and dual 3090 felt surprisingly usable once I tuned around embeddings.

Feel free to tear it apart if you see something dumb.