human evals on small teams

[audioriver]

A couple things I'd check first. If you're on LM Studio, I'd confirm the version first and then reduce variables until the behavior is obvious.

I'd approach it in a boring order. If you're on TensorRT-LLM, I'd confirm the version first and then reduce variables until the behavior is obvious. Not glamorous advice, but repeatable baselines save a lot of time.

[cobaltspark]

[quote]A couple things I'd check first. If you're on LM Studio, I'd confirm the version first and then reduce variables until the behavior is obvious.

What exact setup are you running? What model, what runtime, and roughly what context size?

[pixelspark]

Interesting, I hadn't tried that. I don't think your conclusion is far off. The biggest change for me was simplifying the stack instead of adding one more optimization layer.

[modbench]

Thanks, that helps. I don't think your conclusion is far off. The biggest change for me was simplifying the stack instead of adding one more optimization layer.

[southspark]

What exact setup are you running? Is this local-only or are you routing anything externally?

[relayforge]

Thanks, that helps. I don't think your conclusion is far off.