+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Prompting & Context Engineering (https://www.datasculpting.ai/inference/forumdisplay.php?fid=15)
+--- Forum: Prompts / System Prompts (https://www.datasculpting.ai/inference/forumdisplay.php?fid=62)
+--- Thread: few-shot when you actually need it: setup advice (/showthread.php?tid=1377)
Pages:
1
2
few-shot when you actually need it: setup advice - mintstation99 - 10-17-2026
Did a bit of testing and thought I'd post the result here.
I tried this with Phi using MLX on 4090. The headline result was better than I expected, but speculative decoding still ended up being the limiting factor.
Nothing scientific here, just practical notes from a real setup.
My use case is pretty ordinary: local chat, some document work, and occasional batch jobs.
The part that surprised me was how much the boring details mattered more than the model choice.
Re: few-shot when you actually need it: setup advice - northgarden37 - 10-17-2026
Quote:Did a bit of testing and thought I'd post the result here.
A couple questions before I guess too hard. Are you testing this on 4090 or something else?
Re: few-shot when you actually need it: setup advice - mlxgarden - 10-18-2026
same here on my side.
Re: few-shot when you actually need it: setup advice - audiopoint18 - 10-18-2026
Appreciate the detail. That matches what I saw on A6000. I got better results once I stopped chasing one metric and just looked at whether the workflow felt usable. At this point the real bottleneck is my patience.
Re: few-shot when you actually need it: setup advice - quiettoken - 10-18-2026
That matches what I was seeing. I don't think your conclusion is far off. I got better results once I stopped chasing one metric and just looked at whether the workflow felt usable.
Re: few-shot when you actually need it: setup advice - mlxgarden - 10-18-2026
A couple things I'd check first. Before rebuilding anything, I'd test one clean baseline with Mixtral and compare from there.
Re: few-shot when you actually need it: setup advice - northgarden37 - 10-18-2026
following this because I'm in the same boat.
Re: few-shot when you actually need it: setup advice - mlxgarden - 10-19-2026
A couple things I'd check first. Before rebuilding anything, I'd test one clean baseline with Phi and compare from there.
Re: few-shot when you actually need it: setup advice - marbleyard32 - 10-19-2026
Ran a quick check on my box. The issue went away once I changed one setting and restarted the stack cleanly. I expected the model choice to matter more than it did.
Re: few-shot when you actually need it: setup advice - quiettoken - 10-19-2026
Did a small comparison this afternoon. The issue went away once I changed one setting and restarted the stack cleanly. The main thing that moved was latency, not quality.