When your Spark outputs garbage, it's usually the FP4 kernel — not your prompt
Field note · measured on GB10 / sm_121
You load a quantized model, send a normal prompt, and get back word salad — or a line of !!!!!, or fluent text that's subtly, confidently wrong. You retry with different prompts, different sampling, and it persists. The instinct is "the model is broken" or "my prompt is bad." On GB10 it's almost always neither: it's the quantization kernel silently doing the wrong math.
Why "it runs" and "it runs correctly" are different claims on this chip
GB10 is compute capability sm_121. Many FP4 and FP8 kernels were written and tuned for other Blackwell-class chips (sm_100, sm_120) — and here's the trap: on GB10 those kernels frequently pass their own is_supported() check, dispatch, and produce output. The output is just garbage — row-identical nonsense, or all-NaN logits that never raise an exception. The pipeline reports success at every stage. Nothing errors. The tokens are simply wrong.
The tell: output that's structurally broken (repeated characters, "!!!!!", identical rows) rather than merely low-quality is a kernel signature, not a model or prompt problem. If FP8 gives you coherent text and FP4 gives you noise on the same model, you've found a kernel dispatch bug, not a quantization-quality tradeoff.
The counterintuitive part: nvfp4 is often a trap on Spark today
nvfp4 is the "best" format on paper and the one everyone reaches for — and on GB10 it frequently falls back to a slow dequantization path or dispatches a broken kernel, to the point where a correctly-configured FP8 can be both faster and actually correct. Which format is safe, on which framework version, with which backend flags, changes as the ecosystem patches sm_121 support — and the published benchmark numbers you're comparing against were often produced on a custom dev branch that doesn't match what you installed.
What to do before you blame the model
- Structurally-broken output = suspect the kernel first. Check which quantization backend actually dispatched, not which one you configured.
- Test a known-good format as a control. If FP8 is coherent and FP4 is garbage on the same weights, the model is fine.
- Don't trust
is_supported()on sm_121. Passing the check is not the same as producing correct numbers on this chip.
Which format and backend give you correct and fast output on your framework version is a moving target that has burned a lot of people into thinking their model or their hardware is defective. It's neither — it's a kernel-dispatch problem with a knowable answer.
Getting garbage out of a model that should work?
We determine whether it's a kernel-dispatch bug or a real model issue — on real GB10 hardware, with a known-good control — and tell you the format and backend that give you correct, fast output. Or it's free.
Book a 1-day diagnostic →Free to read and share. Numbers are from hardware we've measured directly; your versions and topology will differ — measure your own.