Inference › Training & Customization › Model Merging
vector DB that stayed out of the way

Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Threaded Mode
vector DB that stayed out of the way
torchsupply99 Offline
Junior Member
**
Posts: 22
Threads: 3
Joined: Sep 2026
Reputation: 0
#11
09-19-2026, 02:15 PM
subscribed, curious where this lands.
learning in public
Find
Reply
syshouse Offline
Junior Member
**
Posts: 33
Threads: 5
Joined: Aug 2026
Reputation: 0
#12
09-20-2026, 02:52 AM
Appreciate the detail. I ended up in about the same place using TensorRT-LLM.
Find
Reply
rasterbench Offline
Member
***
Posts: 73
Threads: 8
Joined: Sep 2026
Reputation: 0
#13
09-20-2026, 03:04 PM
Interesting, I hadn't tried that. I don't think your conclusion is far off.
Find
Reply
mlxgarden Offline
Member
***
Posts: 199
Threads: 27
Joined: Sep 2026
Reputation: 0
#14
09-20-2026, 03:25 PM
Quote:Interesting, I hadn't tried that. I don't think your conclusion is far off.

following this because I'm in the same boat.
M3 Max • MLX • local-first
Find
Reply
routecraft Offline
Member
***
Posts: 193
Threads: 27
Joined: Aug 2026
Reputation: 0
#15
09-20-2026, 04:03 PM
What worked for me was pretty simple. On B200, the easiest win is often to back off the aggressive settings and see whether the problem survives. Not glamorous advice, but repeatable baselines save a lot of time.
routing beats arguing over one perfect model
Find
Reply
vrambandit Offline
Member
***
Posts: 168
Threads: 14
Joined: Sep 2026
Reputation: 0
#16
09-21-2026, 04:26 PM
following this because I'm in the same boat.
2x3090 • llama.cpp • too much VRAM is still not enough
Find
Reply


Forum Jump:


Users browsing this thread: 1 Guest(s)