Jonna Matthiesen

Agreed on the numbers, they're in the article. The table lists 36 blocks × 2048 vs 64 blocks × 1536, and the depth-pinning is called out explicitly: "because the expert shadows the backbone block for block, its depth is not a free parameter." The post one-liner compresses that to "remains at 2.3B," which is a parameter-count claim, not a shape claim. Fair to note 2.28B → 2.42B is a ~6% drift under the rounding.

One thing was held fixed though: the attention interface. 16 × 128 = 2048 in both generations, sized to the backbone's KV head geometry rather than the expert's own hidden width, see the dashed overhang in Figure 2. It's also the part relevant to your question. Width-prune the backbone (hidden, FFN, query heads) and the expert carries over unchanged; depth-prune and architecture breaks. The deployment section draws exactly that line.

Good catch on Parakeet and thanks for your comment(s), really appreciate the discussion! I brought this up with the team who ran the benchmarks on Parakeet, they should drop into this conversation soon 😄.

And yes, for Parakeet, the headline should indeed be focused around the memory saving instead of speedup. I will update the visuals for this model on the page.

Adding memory to our benchmarking script in general seems reasonable, we will add it to future benchmarks.👍

Good points! The order is a bit project-dependent though, and where the interest is strongest. From my point of view, Qwen3-1.7B-W4A16 is a bit more interesting as I see a bit more use around it and it can move an actual deploy decision.

But it is not necessary about which accuracy row lands next, but which model x hardware combination we optimize next. Being able to replace an existing deployed model fast with a newer, better one is quite a strong argument, or moving to a cheaper hardware.

In terms of what models may come next, I could see VLAs becoming more interesting going forward since low-latency real-time deployment scenarios are key there.
For example, we recently did a project focusing on VLA (π0.5) optimization on AMD. Some of this was in the AMD keynote last week. We did the GPU VLA inference-latency optimization behind their sub-100ms robot-reasoning number on the Kria AI SOM.

And FYI, missing accuracy benchmarks for the non-LLM/VLM models have been added now. 😊

Some models (like https://huggingface.co/embedl/sam3) have more benchmarks in the model page - will push for updating the Benchmarks page with those results.

Yes, accuracy numbers are a bit thin! For FlashHead you are correct. We have written a paper on FlashHead (see https://arxiv.org/abs/2603.14591) where we show that it is consistently accuracy-preserving: typically within ~99.9% top-1 containment.

For the GenAI models on Jetson: We are actually using the TensorRT vLLM Docker image, so lm-eval has a path and is not blocked by the TRT engine. For example, for Qwen3.6 we have internal accuracy evaluations but not yet pushed.

For the more classical DL models, the benchmark suite mostly compares to quantized variants where we do need to push for more on-device accuracy evaluations.

The notion of picking the largest model that fits the latency (and memory) budget is quite common and that's one focus of our GenAI models . On the classical DL side it is a bit different, edge deployment use-cases are often around making an existing, well performing model faster.

Any plans on releasing flashhead for qwen3.5 models?

Yes! FlashHead-enabled Qwen3.5 models are coming soon. We are currently finalizing accuracy and latency evaluations.

Check it out here: https://github.com/JonnaMat/huggingface-slack-app

🤓 Want to learn more about FlashHead? Check out this blog post: https://huggingface.co/blog/JonnaMat/flashhead