Inference › Inference Engines › llama.cpp / GGUF v
multi-GPU offload advice

Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Linear Mode
multi-GPU offload advice
embershed99 Offline
Member
***
Posts: 56
Threads: 7
Joined: Sep 2026
Reputation: 0
#10
09-01-2026, 12:27 PM
Quote:I'd approach it in a boring order. If you're on SGLang, I'd confirm the version first and then reduce variables until the behavior is obvious.

Interesting, I hadn't tried that. I don't think your conclusion is far off. Worth rechecking your defaults though, because one bad assumption can make the rest of the thread look confusing.
Find
Reply


Messages In This Thread
multi-GPU offload advice - by bytesupply76 - 08-31-2026, 04:59 AM
Re: multi-GPU offload advice - by dockridge - 08-31-2026, 05:52 AM
Re: multi-GPU offload advice - by phasescope - 08-31-2026, 06:01 AM
Re: multi-GPU offload advice - by thetagrid15 - 08-31-2026, 07:18 AM
Re: multi-GPU offload advice - by macroforge - 08-31-2026, 08:04 AM
Re: multi-GPU offload advice - by bytesupply76 - 08-31-2026, 10:50 AM
Re: multi-GPU offload advice - by embershed99 - 09-01-2026, 12:27 PM
Re: multi-GPU offload advice - by llm_wren - 08-31-2026, 11:13 AM
Re: multi-GPU offload advice - by ggufwire22 - 08-31-2026, 12:04 PM
Re: multi-GPU offload advice - by sparrowkite53 - 08-31-2026, 12:20 PM
Re: multi-GPU offload advice - by northcraft90 - 09-01-2026, 12:37 PM

Forum Jump:


Users browsing this thread: 1 Guest(s)