DFlash on ARC B70 - Finaly

2026-09-13 · 1 min read

Recent versions of vLLM seem to finaly ship enough fixes so my 2x Intel B70 Pro setup now finaly can run Qwen 3.8 27B + DFlash.

Performance improvement is significant, from standard ~19-22 tokens/second to anywhere from 40 to 140 tokens/s. Output performance greatly depends on how draft model is able to predict output, it performs much better on code or structured output than language and general text.

Full command line:

vllm serve \
  Qwen/Qwen3.8-27B \
  --safetensors-load-strategy eager \
  --tensor-parallel-size 2 \
  --trust-remote-code \
  --served-model-name Qwen3.8-27B \
  --gpu-memory-utilization 0.96 \
  --enable-auto-tool-choice \
  --tool-call-parser qwen3_xml \
  --reasoning-parser qwen3 \
  --max-model-len auto \
  \
  --kv-cache-dtype fp8 \
  --quantization fp8 \
  --dtype bfloat16 \
  --language-model-only \
  \
  --override-generation-config '{"temperature": 1.0, "top_k": 20, "top_p": 0.95, "min_p": 0.0, "presence_penalty": 0.0, "repeat_penalty": 1.0}' \
  --default-chat-template-kwargs '{"preserve_thinking": true, "reasoning_effort": "medium"}' \
  --speculative-config '{"method": "dflash", "model": "incoai/Qwen3.8-27B-DFlash2", "num_speculative_tokens": 7}' \
  --compilation-config '{"max_cudagraph_capture_size": 64}' \
  \
  --max-num-seqs 8 \
  --max-num-batched-tokens 1024 \
  --optimization-level 3 \