10-07-2026, 05:40 PM
Interesting, I hadn't tried that. I ended up in about the same place using vLLM. The biggest change for me was simplifying the stack instead of adding one more optimization layer.
Inference
›
Drivers / Frameworks / Toolkits
›
NVIDIA / CUDA
Tensor cores actually helping here for Gemma
Tensor cores actually helping here for Gemma
|
Tensor cores actually helping here for Gemma
|
|
« Next Oldest | Next Newest »
|
| Messages In This Thread |
|
Tensor cores actually helping here for Gemma - by phaseport36 - 10-07-2026, 09:02 AM
Re: Tensor cores actually helping here for Gemma - by orbitbench6 - 10-07-2026, 05:40 PM
Re: Tensor cores actually helping here for Gemma - by vectorember - 10-07-2026, 06:13 PM
Re: Tensor cores actually helping here for Gemma - by dockember - 10-10-2026, 07:14 PM
Re: Tensor cores actually helping here for Gemma - by orbitbench6 - 10-07-2026, 06:56 PM
Re: Tensor cores actually helping here for Gemma - by thetapath - 10-11-2026, 07:56 PM
Re: Tensor cores actually helping here for Gemma - by phaseport36 - 10-13-2026, 08:37 AM
Re: Tensor cores actually helping here for Gemma - by marbletrack - 10-14-2026, 08:51 AM
|
Users browsing this thread: 1 Guest(s)