https://www.datasculpting.ai/inference/ Mon, 26 Oct 2026 11:25:20 +0000 MyBB https://www.datasculpting.ai/inference/showthread.php?tid=1501 Mon, 19 Oct 2026 05:25:13 +0000 Data Sculpting]]> https://www.datasculpting.ai/inference/showthread.php?tid=1501
https://github.com/DataSculpting/VisLlama

It's not just a UI, it allows juggling of multiple engine build presets, multiple model presets, allows everything manageable via UI, GPU assignment including priority and typical splits, auto-launch, launch order. It's everything LM Studio or Ollama should be but aren't. It also puts ALL loaded models (regardless of engine) behind a common OpenAI-compatible endpoint and has separate router management options. Built-in chat, system stats, historical inference stats, etc.]]>
https://github.com/DataSculpting/VisLlama

It's not just a UI, it allows juggling of multiple engine build presets, multiple model presets, allows everything manageable via UI, GPU assignment including priority and typical splits, auto-launch, launch order. It's everything LM Studio or Ollama should be but aren't. It also puts ALL loaded models (regardless of engine) behind a common OpenAI-compatible endpoint and has separate router management options. Built-in chat, system stats, historical inference stats, etc.]]> https://www.datasculpting.ai/inference/showthread.php?tid=1015 Sat, 10 Oct 2026 20:19:02 +0000 oxiderunner]]> https://www.datasculpting.ai/inference/showthread.php?tid=1015
Two paths both seem viable, but they break down differently once workstation thermals becomes important.

Interested in tradeoffs more than fan loyalty.

This is mostly for local work rather than public serving.

The part that surprised me was how much the boring details mattered more than the model choice.]]>
Two paths both seem viable, but they break down differently once workstation thermals becomes important.

Interested in tradeoffs more than fan loyalty.

This is mostly for local work rather than public serving.

The part that surprised me was how much the boring details mattered more than the model choice.]]> https://www.datasculpting.ai/inference/showthread.php?tid=344 Sat, 19 Sep 2026 00:02:43 +0000 sparrowworks88]]> https://www.datasculpting.ai/inference/showthread.php?tid=344
This isn't revolutionary, but the combination of LM Studio and M3 Max felt surprisingly usable once I tuned around continuous batching.

If someone wants the exact settings I can clean them up and post them.]]>
This isn't revolutionary, but the combination of LM Studio and M3 Max felt surprisingly usable once I tuned around continuous batching.

If someone wants the exact settings I can clean them up and post them.]]> https://www.datasculpting.ai/inference/showthread.php?tid=248 Sun, 13 Sep 2026 05:03:51 +0000 batchember]]> https://www.datasculpting.ai/inference/showthread.php?tid=248
My current setup is SGLang with 5090, and I keep running into reranking.

Not looking for the perfect answer, mostly what worked in real use.]]>
My current setup is SGLang with 5090, and I keep running into reranking.

Not looking for the perfect answer, mostly what worked in real use.]]>