3 Ways to Boost MoE Inference on NVIDIA DGX Spark

Profiling a batch-size-1 evaluation of Inclusion AI's Ling-3.0-flash on NVIDIA DGX Spark showed decoding was memory-bound, with model weight reads driving execution time. The MoE computation and the output-layer projection that produces scores across the model stood out as the two key operations.