A local language model usually generates a response one token at a time. Longer responses mean more sequential generation steps. Speculative decoding changes the execution path: a cheaper draft model proposes several tokens, and the target model verifies them together. When enough candidates survive verification, the target needs fewer sequential passes.

This article uses SparkLLM’s Spark-X2.6-4B and its DSpark draft model to explain the mechanism and how to interpret its benchmark. It assumes familiarity with running language models, not with speculative decoding. Performance numbers are from the project’s model card, not an independent benchmark; sources were checked on November 7, 2026.

Generating candidates and checking them

A token is a unit of text processed by a language model. In ordinary autoregressive decoding, the model predicts a token from the existing context, appends it, and repeats. Each step depends on the previous result.

Speculative decoding divides the work between a target model, which defines the final generation behavior, and a draft model, which proposes a sequence. Because the candidate sequence is already available, the target can compute verification information for its positions in one forward pass. The decision about which candidates to keep is still ordered, but the target need not run a separate pass for each candidate.

For a greedy-decoding example, suppose the draft proposes A B C D and the target agrees on the first two positions but chooses X at the third:

Draft:        A  B  C  D
Verification: ✓  ✓  ✗
Output:       A  B  X

Once C is rejected, D also has to be discarded: it was proposed under a prefix that no longer holds. The next round continues from A B X. If all candidates are accepted, the target can also provide an extra token conditioned on the complete candidate sequence. An end token or output limit can stop generation earlier.

That example describes greedy decoding. With random sampling, verification is not simply a comparison of two independently sampled tokens. Standard speculative sampling uses acceptance probabilities and a correction distribution after rejection to preserve the target’s sampling distribution. Matching distributions does not guarantee identical text for the same random seed; the guarantee also depends on the engine’s sampling implementation and configuration. The original speculative decoding paper explains the method.

The Spark model pair

Spark-X2.6-4B-GGUF is the target; Spark-X2.6-4B-DSpark-GGUF supplies the draft. The latter is paired with this target and does not replace it in the inference path.

The draft model card lists 624 million parameters, five sliding-window attention layers with a window of 512, and BF16 draft weights. The target can use BF16 or quantized GGUF weights. DSpark uses intermediate representations, or hidden states, from target layers 2, 10, 18, 25 and 33 to help generate candidates. Each draft forward pass proposes up to seven tokens for target verification.

The window limits the positions directly attended to by those attention layers. Total inference cost also includes the target, caches and other operations, so that window does not make overall cost independent of context length. The draft itself consumes memory and compute; adding it is not automatically a speedup.

Acceptance length and speed measure different things

The model card measures mean acceptance length with seven draft tokens and a maximum output length of 4,096 tokens. Its acceptance length includes one additional target token, so the maximum is eight. An average of four therefore does not mean four draft tokens were accepted per round.

Thinking mode, target temperature or draft sampling change across these columns, so the differences cannot be attributed to any one setting alone. Within the first column, MATH500 reaches 4.59 and Arena-Hard-v2 3.22: useful draft length varies by workload in this evaluation.

More output tokens per round can amortize target-model work, but drafting and verification both take time. A useful approximation is:

Speedup ≈ average output tokens per round × baseline time per token ÷ speculative time per round.

An average of about four output tokens per round is not a fourfold speedup. Early rejections or expensive drafting and verification reduce the benefit and can make execution slower.

Reading the reported throughput

The model card reports mean generation throughput across six datasets, in tokens per second. The RTX 4090 runs use llama.cpp; the M5 Pro runs use MLX. Because both hardware and runtime differ, their results cannot isolate a hardware effect. The quantization formats differ too.

These are four configuration averages, not bounds for every task. For example, the M5 Pro BF16 result is 2.94× on MATH500, while the M5 Pro INT4 result is 1.30× on Arena-Hard-v2. Throughput and speedup values are rounded, so ratios calculated from the displayed numbers may differ slightly.

Generation throughput is not the same as time to first token or end-to-end request latency. Prompt processing, output length, concurrency, sampling settings, available memory and implementation version can all change the result. Quantizing the target also makes baseline decoding faster; whether drafting still pays for itself has to be measured on the intended device.

Testing the tradeoff locally

DSpark is a candidate for workloads that generate substantial amounts of text when the device can hold the target, BF16 draft weights and runtime caches together. For short outputs, prompt-dominated requests or constrained memory, measure total latency and memory before deciding to enable it.

For an on/off comparison, keep the target and its quantization, prompts, output length, sampling and thinking settings, context length, and concurrency consistent. Record warmed-up generation tokens per second, time to first token, end-to-end latency and peak memory. Repeat the measurements on representative tasks. Speculation changes how random numbers are consumed, so sampled responses can differ; control the workload rather than requiring identical text.

As of the check date, the model card’s GGUF instructions use the master branch of XHToken/llama.cpp on Linux, Bash and NVIDIA GPUs, and say upstream integration is in progress. The M5 Pro results use MLX; the CUDA instructions are not an MLX setup. Consult the model card for current versions and instructions. Ordinary inference requires the target alone; this speculative path also needs the matching draft weights and a supporting runtime.

Sources:

• Spark-X2.6-4B-DSpark-GGUF model card (SparkLLM, undated; checked 2026-11-07)
• Spark-X2.6-4B-GGUF model card (SparkLLM, undated; checked 2026-11-07)
• Spark-X2.6-4B DSpark Is Here (SparkLLM, 2026-10-31)
• Fast Inference from Transformers via Speculative Decoding (Leviathan et al., 2023)