Paper Review
Fast-dLLM: Training-Free Acceleration of Diffusion LLMs via KV Cache and Parallel Decoding
A block-wise approximate KV cache plus confidence-thresholded parallel decoding accelerates open diffusion LLMs by up to 27.6x end-to-end with 1-2 point accuracy loss, entirely training-free.
Block 단위 approximate KV cache와 confidence threshold 기반 parallel decoding으로, 재학습 없이 open diffusion LLM 추론을 최대 27.6배 가속하면서 정확도 하락을 1-2pt 이내로 유지한다.
Background Diffusion LLMs (LLaDA, Dream) promise non-autoregressive parallel generation, yet the measured inference speed of open models lags behind KV-cached autoregressive models. At the closed frontier the promise is already real — Mercury reports over 1,000 tokens/s and Google DeepMind's Gemini Diffusion over 1,400 tokens/s — but open checkpoints fall short of even ordinary AR throughput. The paper traces this gap to two structural causes: bidirectional attention blocks KV caching, and multi-token parallel decoding degrades quality, so in the quality-preserving regime an open dLLM effectively commits one token per full-sequence forward pass.
배경 Diffusion LLM(LLaDA, Dream)은 non-autoregressive parallel 생성을 약속하지만, 공개 모델의 실측 추론 속도는 KV cache를 쓰는 AR 모델보다 오히려 느리다. Closed 진영에서는 이 약속이 이미 현실이다 — Mercury는 1,000+ tokens/s, Google DeepMind의 Gemini Diffusion은 1,400+ tokens/s를 보고한다 — 그런데 공개 checkpoint는 평범한 AR throughput에도 못 미친다. 논문은 이 격차를 두 구조적 원인으로 추적한다: bidirectional attention이 KV caching을 막고, multi-token parallel decoding은 품질을 무너뜨려, 품질을 지키는 구간에서는 사실상 full-sequence forward 한 번에 token 하나만 확정하게 된다.
Problem Bidirectional attention rules out the standard AR KV cache, and naively committing many tokens per step sharply degrades generation quality. Without a cache, every denoising step recomputes attention over the entire sequence, so an architecture built for parallelism ends up slower than the AR models it competes with. The quality side is what the paper calls the "curse of parallel decoding": simultaneously committed tokens are drawn from independent marginals that ignore joint dependencies — like several writers each filling in one blank of the same sentence without seeing each other's choices, every word plausible alone but the sentence incoherent jointly.
문제 Bidirectional attention 때문에 표준 AR KV cache를 쓸 수 없고, 한 step에 여러 token을 naive하게 동시 확정하면 생성 품질이 급락한다. Cache가 없으니 매 denoising step마다 전체 sequence에 대한 attention을 재계산해야 해서, parallelism을 위해 설계된 아키텍처가 경쟁 상대인 AR 모델보다 오히려 느려진다. 품질 쪽 문제가 논문이 말하는 "curse of parallel decoding"이다: 동시 확정되는 token들은 independent marginal에서 뽑혀 joint 의존성을 무시한다 — 여러 사람이 서로의 답을 못 본 채 같은 문장의 빈칸을 하나씩 채우는 것과 같아서, 단어 하나하나는 그럴듯해도 문장 전체는 어긋난다.
Prior limitations Prior dLLM inference recomputed full attention at every denoising step, and fixed token-per-step schedules break inter-token dependencies under the conditional independence assumption — identified here as the root cause of parallel-decoding quality loss. LLaDA's default top-K scheme commits K tokens per step whether the model is confident or not, which is why accuracy falls as K grows (the top-2/4/8 baselines in Figure 5). Neither bottleneck had a training-free fix: caching was considered incompatible with bidirectional attention, and parallelism existed only as a fixed, non-adaptive schedule.
기존 한계 기존 dLLM 추론은 매 denoising step마다 전체 attention을 재계산했고, 고정 token-per-step 스케줄은 conditional independence 가정 아래 token 간 의존성을 깨뜨린다는 것이 품질 저하의 근본 원인으로 지목된다. LLaDA의 기본 top-K 방식은 모델이 확신하든 말든 매 step 똑같이 K개를 확정하므로 K가 커질수록 정확도가 떨어진다(Figure 5의 top-2/4/8 baseline). 결국 두 병목 모두 training-free 해법이 없었다 — caching은 bidirectional attention과 양립 불가로 여겨졌고, parallelism은 적응형이 아닌 고정 스케줄뿐이었다.
Goal Bring dLLM inference throughput close to AR levels training-free, without touching model weights and with minimal accuracy loss. Both bottlenecks must fall together — reusing computation across steps (caching) and cutting the number of steps (parallel commits) — because fixing only one leaves the other dominating runtime, as the ablation later confirms (3.2× and 2.5× alone vs 8.1× combined). The constraint is strict drop-in deployment: existing LLaDA/Dream checkpoints, with changes confined to the decoding loop.
목표 모델 가중치를 건드리지 않는 training-free 방식으로, 정확도 손실을 최소화하며 dLLM 추론 throughput을 AR 수준에 근접시키는 것. 두 병목은 함께 무너뜨려야 한다 — step 간 계산 재사용(caching)과 step 수 자체의 절감(parallel commit) — 하나만 고치면 나머지가 runtime을 지배하며, 이는 뒤의 ablation(단독 3.2×·2.5× vs 결합 8.1×)이 확인해 준다. 제약은 엄격한 drop-in 배포다: 기존 LLaDA/Dream checkpoint 그대로, 수정은 decoding loop 안에서만.
Method Generation proceeds block by block with prefix KV activations cached and refreshed only at block boundaries (the DualCache variant also caches the masked suffix), combined with confidence-aware parallel decoding that commits only tokens whose max-softmax confidence exceeds a threshold τ (default 0.9), with a highest-confidence fallback to guarantee progress. After a block completes, the caches of all blocks are recomputed once, so staleness is bounded by the block size (default 32, swept 4-32; threshold swept 0.5-1.0). The cache treats the rest of the sequence like the frozen background of a film set: only the active block is live, and the background is re-shot once per block rather than once per frame. An appendix variant, factor decoding, adaptively picks the largest n satisfying (n+1)(1−c⁽ⁿ⁾) < f, tying the parallelism level directly to the theorem's bound below.
방법 생성을 block 단위로 진행하며 prefix(및 DualCache 변형에서는 masked suffix까지)의 KV activation을 캐시해 block 경계에서만 갱신하고, 각 step에서는 max-softmax confidence가 threshold τ(기본 0.9)를 넘는 token만 선별적으로 확정하는 confidence-aware parallel decoding(전부 미달 시 최고-confidence token은 항상 확정해 진행 보장)을 결합한다. Block 하나가 끝나면 전체 block의 cache를 한 번 재계산하므로 staleness의 상한이 block size(기본 32, 4-32 sweep; threshold는 0.5-1.0 sweep)로 잡힌다. Cache는 나머지 sequence를 영화 세트의 고정 배경처럼 다루는 셈이다: 활성 block만 live이고, 배경은 frame마다가 아니라 block마다 한 번씩만 다시 찍는다. 부록의 factor decoding 변형은 (n+1)(1−c⁽ⁿ⁾) < f를 만족하는 최대 n을 골라 parallelism 정도를 아래 정리의 bound에 직접 연동한다.
Key idea KV activations outside the active block are nearly static across adjacent denoising steps — the cosine-similarity heatmaps of Figure 3 concentrate near the diagonal for both prompt and suffix blocks — so approximate reuse is safe. Theorem 1 then proves greedy parallel decoding exactly matches greedy sequential decoding whenever each of the n committed tokens carries confidence above 1−1/(n+1), i.e. (n+1)·ε ≤ 1, and shows this bound is tight. The bar rises with the crowd: committing more tokens at once demands that each be closer to certain — like letting several people speak simultaneously only when each is reciting a foregone conclusion. A second clause bounds the distributional distance between the true joint and the factorized product, covering the non-greedy case as well.
핵심 아이디어 활성 block 밖 KV activation은 인접 denoising step 간 거의 정적이다 — Figure 3의 cosine similarity heatmap이 prompt·suffix block 모두에서 대각선 근처에 몰려 있다 — 따라서 approximate 재사용이 안전하다. Theorem 1은 확정되는 n개 token 각각의 confidence가 1−1/(n+1)을 넘으면, 즉 (n+1)·ε ≤ 1이면 greedy parallel decoding이 greedy sequential decoding과 정확히 일치함을 증명하고 이 bound가 tight함까지 보인다. 인원이 늘수록 기준선도 올라간다: 한 번에 더 많은 token을 확정하려면 각각이 더 확실해야 한다 — 여러 사람이 동시에 말해도 되는 경우란 각자가 이미 정해진 결론을 읊고 있을 때뿐인 것과 같다. 두 번째 절은 true joint와 factorized product 간 분포 거리까지 bound해 non-greedy 경우도 커버한다.
Validation Evaluated LLaDA/LLaDA-1.5/Dream (plus multimodal LLaDA-V) on GSM8K (5-shot), MATH (4-shot), HumanEval (0-shot), MBPP (3-shot) (+MathVista/MathVerse), generation lengths 256-1024, single A100 80GB, using the standardized lm-eval library. Throughput is measured end-to-end as average output tokens per second until <eos>, at batch size 1 with no serving frameworks — the numbers reflect raw decoding-loop gains. Component-wise results (+Cache, +Parallel, +both) are reported separately across every benchmark and length, isolating the two contributions rather than showing only the combined figure.
검증 LLaDA/LLaDA-1.5/Dream(및 multimodal LLaDA-V)을 GSM8K(5-shot)/MATH(4-shot)/HumanEval(0-shot)/MBPP(3-shot)(+MathVista/MathVerse)에서 생성 길이 256-1024, 단일 A100 80GB, 표준 lm-eval 라이브러리 기반으로 평가했다. Throughput은 <eos>까지의 평균 출력 token/s로 end-to-end 측정하며, batch size 1에 serving framework 없이 재는 순수 decoding-loop 이득이다. 모든 벤치마크·길이에서 성분별 결과(+Cache, +Parallel, +둘 다)를 따로 보고해, 결합 수치만이 아니라 두 기여를 분리해 보여준다.
Results Up to 27.6× end-to-end throughput (LLaDA, 8-shot prefill, 1024 tokens) and 11.0× on GSM8K-512, with accuracy within 1-2 points (GSM8K-256: 79.3 → 78.5%); component-wise, KV cache alone gives 3.2×, parallel decoding 2.5×, combined 8.1× (6.7 → 54.4 tokens/s). Gains grow with prefill and generation length — exactly where the recompute burden is heaviest — with DualCache benefiting most. On multimodal LLaDA-V, a refresh-based full-block variant reaches 9.9× on MathVista with MathVerse accuracy even slightly improved (28.5 → 28.6), and the appendix factor-decoding variant pushes GSM8K-256 to 11.7× at 77.5% accuracy.
결과 최대 27.6× end-to-end throughput 향상(LLaDA, 8-shot prefill, 1024 token), GSM8K-512에서 11.0×, 정확도는 GSM8K-256 기준 79.3→78.5%로 1-2pt 이내 하락, 단독 성분은 KV cache 3.2×·parallel decoding 2.5×·결합 8.1×(6.7→54.4 token/s)다. 이득은 prefill과 생성 길이가 길수록 — 즉 재계산 부담이 가장 무거운 곳에서 — 커지며, DualCache가 가장 크게 혜택을 본다. Multimodal LLaDA-V에서는 refresh 기반 full-block 변형이 MathVista 9.9×를 달성하고 MathVerse 정확도는 오히려 소폭 상승(28.5→28.6)하며, 부록의 factor decoding은 GSM8K-256을 정확도 77.5%에서 11.7×까지 끌어올린다.
Comparison Compared against vanilla LLaDA/Dream decoding and fixed top-K token-per-step baselines, showing the two components compose multiplicatively (3.2× · 2.5× → 8.1×). Figure 5 shows the confidence threshold dominating fixed top-2/4/8 schedules: adaptive parallelism commits many tokens on easy steps and throttles on hard ones, holding accuracy where fixed-K degrades. Appendix C.5 adds an honest cross-family comparison against AR LLaMA under varying batch sizes: comparable or better at small batch, overtaken as batch grows.
비교 가속 없는 vanilla LLaDA/Dream decoding과 고정 top-K token-per-step 베이스라인 대비 평가했으며, 두 성분이 상보적으로 곱해짐(3.2×·2.5× → 8.1×)을 보인다. Figure 5는 confidence threshold가 고정 top-2/4/8 스케줄을 지배함을 보여준다: adaptive parallelism은 쉬운 step에서 많이 확정하고 어려운 step에서 절제해, 고정 K가 무너지는 곳에서도 정확도를 지킨다. 부록 C.5는 batch size를 바꿔가며 AR LLaMA와 직접 비교하는 정직한 교차-계열 실험까지 싣는다: 작은 batch에서는 대등하거나 우세, batch가 커지면 역전당한다.
Significance The first widely adopted recipe that makes open dLLM inference practically competitive with KV-cached AR decoding without retraining, formalizing cache staleness and adaptive parallelism as measurable tuning knobs and becoming the standard baseline for subsequent dLLM acceleration work (reviewer judgment). The deeper implication is that dLLM slowness was a decoding-loop property, not a weights property: two interventions that never touch the model recover an order of magnitude. Open dLLMs were not slow because the engine was weak — they shipped with two handbrakes on (full recompute every step, one token per step), and this paper releases both.
의의 Open dLLM 추론을 재학습 없이 KV-cached AR decoding과 실질적으로 경쟁 가능하게 만든 첫 대표 사례로, cache staleness와 adaptive parallelism을 측정 가능한 조절 변수로 정식화해 이후 dLLM 가속 연구의 표준 베이스라인이 되었다(리뷰어 판단). 더 깊은 함의는 dLLM의 느림이 weights의 속성이 아니라 decoding loop의 속성이었다는 점이다: 모델을 전혀 건드리지 않는 개입 둘이 한 자릿수 배율을 회복한다. Open dLLM은 엔진이 약해서 느렸던 게 아니라 핸드브레이크 두 개(매 step 전체 재계산, step당 한 token)를 채운 채 출고된 것이었고, 이 논문은 그 둘을 풀어준 셈이다.
Limitations The paper itself notes that at larger batch sizes full-attention overhead lets AR LLaMA overtake it in throughput, and that multimodal LLaDA-V is block-size sensitive (>8% accuracy drop shrinking blocks 96→8 on MathVista); a systematic accuracy-staleness frontier map remains open (reviewer judgment). The batch mechanism is instructive: as batch grows, AR decoding transitions from memory-bound to compute-bound and recovers GPU utilization, while the dLLM keeps paying full attention over the whole sequence every step — the headline wins are fundamentally small-batch results. Theorem 1's equivalence clause also covers only greedy decoding; sampling regimes get distance bounds, not equivalence.
한계 논문 스스로 큰 batch에서는 full attention 오버헤드로 AR LLaMA throughput에 뒤진다고 명시하고 multimodal LLaDA-V는 block size에 민감하며(MathVista에서 block 96→8 축소 시 >8% 하락), accuracy-staleness frontier의 체계적 지도는 미완이다(리뷰어 판단). Batch 메커니즘이 시사적이다: batch가 커지면 AR decoding은 memory-bound에서 compute-bound로 넘어가며 GPU utilization을 회복하는 반면, dLLM은 매 step 전체 sequence에 대한 full attention 비용을 계속 치른다 — 헤드라인 수치는 본질적으로 small-batch 결과다. 또한 Theorem 1의 equivalence 절은 greedy decoding에만 적용되며, sampling 영역은 equivalence가 아닌 distance bound만 받는다.
Future work No formal future-work section; natural next steps are per-position adaptive thresholds, staleness-aware cache refresh, and composing with AR-style cache compression (quantization/eviction) (reviewer judgment). The batch-scaling gap the authors acknowledge points at serving-level integration — batching strategies and cheaper attention for dLLMs — as the binding follow-up problem (reviewer judgment). The refresh-interval technique used for LLaDA-V (fixed block plus periodic cache refresh) also remains an unexplored scheduling knob awaiting systematic study.
향후 과제 별도 future work 절은 없으며, per-position adaptive threshold·staleness 인지형 cache refresh·AR용 cache 압축(quantization/eviction)과의 결합이 자연스러운 다음 단계다(리뷰어 판단). 저자들이 인정한 batch-scaling 격차는 serving 수준 통합 — dLLM용 batching 전략과 더 싼 attention — 이 다음 병목임을 가리킨다(리뷰어 판단). LLaDA-V에 쓰인 refresh-interval 기법(고정 block + 주기적 cache refresh)도 아직 체계적으로 탐구되지 않은 스케줄링 knob으로 남아 있다.
Released resources Code and project page are released under NVlabs on GitHub (Fast-dLLM); being training-free, no separate checkpoints are required. The method runs directly on public LLaDA, LLaDA-1.5, Dream, and LLaDA-V checkpoints, and evaluation goes through the standardized lm-eval library, so the headline numbers are reproducible end to end with public tooling.
자원 공개 코드와 프로젝트 페이지가 NVlabs GitHub(Fast-dLLM)로 공개되어 있고, training-free 방법이라 별도 checkpoint는 불필요하다. 공개된 LLaDA·LLaDA-1.5·Dream·LLaDA-V checkpoint 위에서 바로 동작하며 평가도 표준 lm-eval 라이브러리를 거치므로, 헤드라인 수치를 공개 도구만으로 끝까지 재현할 수 있다.
The research thread
Where this paper sits 연구의 흐름
Fast-dLLM stands at the junction of two long research lines. The first brought discrete diffusion to text: D3PM (Austin et al., 2021) generalized diffusion to categorical data, SEDD (Lou et al., 2023) and MDLM (Sahoo et al., 2024) simplified masked diffusion into something trainable at scale, and LLaDA (Nie et al., 2025) and Dream 7B finally pushed it to 7-8B parameters with quality worth taking seriously — while closed models like Mercury and Gemini Diffusion proved that 1,000+ tokens/s was physically achievable. The second line is older: non-autoregressive generation out of machine translation, which promised parallel decoding for years and kept paying for it in quality. Block Diffusion (Arriola et al., 2025) had already interpolated between AR and diffusion by retraining for block-causal attention; Fast-dLLM asks how much of that benefit is available with no retraining at all.
The conceptual move is to relocate the slowness. Before this paper the common reading was architectural destiny: bidirectional attention means no KV cache, and conditional independence means parallel commits break the joint — both looked like properties of the model class, fixable only by training a different model. Fast-dLLM reframes both as decoding-loop questions. Cache validity becomes an empirical quantity you measure (KV states outside the active block barely move between adjacent steps) rather than a binary you assume; parallelism becomes a per-step decision the model prices with its own confidence, with Theorem 1's (n+1)·ε ≤ 1 turning a heuristic into a certificate. Staleness and parallelism stop being identity traits and become knobs.
What this opened is visible in how quickly the follow-up literature standardized on it: Fast-dLLM became the de facto baseline for dLLM acceleration work, and 'training-free' became a track of its own. But the paper is candid about where the ledger flips — its own Appendix C.5 shows AR models overtaking as batch size grows, because AR decoding transitions to compute-bound and fills the GPU, while a dLLM keeps paying full attention over the whole sequence every step. The headline 27.6× is a small-batch, single-sequence victory; serving-scale throughput remains unclaimed territory.
So the tension this line inherits is now about scheduling and systems rather than possibility. Fixed block boundaries are a blunt instrument — LLaDA-V swinging by >8% under block-size changes means refresh timing carries real accuracy. The theory has its own overhang: exact equivalence is proven only for greedy decoding, leaving sampling with looser distance bounds. And the deeper question raised by the AR comparison is convergence: block-wise caching makes a dLLM look increasingly like an AR model taking parallel bursts inside blocks, and the field seems headed toward decoders that treat 'AR vs diffusion' as endpoints of one schedule rather than different species.
Fast-dLLM은 두 갈래의 오랜 연구 계보가 만나는 지점에 서 있다. 하나는 discrete diffusion을 텍스트로 가져온 계보다: D3PM(Austin et al., 2021)이 diffusion을 categorical 데이터로 일반화했고, SEDD(Lou et al., 2023)와 MDLM(Sahoo et al., 2024)이 masked diffusion을 대규모 학습이 가능한 형태로 단순화했으며, LLaDA(Nie et al., 2025)와 Dream 7B가 마침내 7-8B 규모에서 주목할 만한 품질을 보여줬다 — 그 사이 Mercury와 Gemini Diffusion 같은 closed 모델은 1,000+ tokens/s가 물리적으로 가능함을 증명했다. 다른 하나는 더 오래된 계보, 기계번역에서 출발한 non-autoregressive generation이다: 수년간 parallel decoding을 약속했지만 늘 품질로 대가를 치렀다. Block Diffusion(Arriola et al., 2025)은 block-causal attention을 재학습해 AR과 diffusion 사이를 보간했는데, Fast-dLLM은 묻는다 — 재학습 없이 그 이득의 얼마만큼을 가져올 수 있는가.
개념적 이동은 '느림의 소재지'를 옮긴 것이다. 이 논문 전까지 통념은 아키텍처의 숙명이었다: bidirectional attention이니 KV cache가 불가능하고, conditional independence니 parallel commit이 joint를 깨뜨린다 — 둘 다 모델 클래스의 속성처럼 보였고, 다른 모델을 학습해야만 고칠 수 있는 문제였다. Fast-dLLM은 둘 다 decoding loop의 질문으로 재정의한다. Cache 유효성은 가정하는 binary가 아니라 측정하는 경험적 양이 되고(active block 밖 KV state는 인접 step 간 거의 움직이지 않는다), parallelism은 모델이 자기 confidence로 값을 매기는 per-step 결정이 되며, Theorem 1의 (n+1)·ε ≤ 1이 heuristic을 certificate로 바꾼다. Staleness와 parallelism이 정체성이 아니라 knob이 된 것이다.
이것이 무엇을 열었는지는 후속 문헌이 얼마나 빨리 이 논문을 표준으로 삼았는지에서 드러난다: Fast-dLLM은 dLLM 가속 연구의 사실상 기본 베이스라인이 되었고, 'training-free'는 독립된 트랙이 되었다. 하지만 논문은 장부가 뒤집히는 지점도 솔직히 적는다 — Appendix C.5에서 batch가 커지면 AR 모델이 다시 앞선다. AR decoding은 compute-bound로 전환되며 GPU를 채우는 반면, dLLM은 매 step 전체 sequence에 대한 full attention 비용을 계속 치르기 때문이다. 헤드라인 27.6×는 small-batch, 단일 sequence의 승리이고, serving 규모의 throughput은 아직 주인 없는 땅이다.
따라서 이 계보가 물려받은 긴장은 이제 가능성이 아니라 스케줄링과 시스템의 문제다. 고정 block 경계는 무딘 도구다 — LLaDA-V가 block size 변화에 >8%씩 흔들리는 것은 refresh 타이밍이 실제 정확도를 쥐고 있다는 뜻이다. 이론에도 미결이 남아 있다: 정확한 equivalence는 greedy decoding에만 증명되었고, sampling은 더 느슨한 distance bound만 받는다. 그리고 AR 비교가 던지는 더 깊은 질문은 수렴이다: block 단위 caching은 dLLM을 'block 안에서 parallel burst를 하는 AR 모델'처럼 보이게 만들며, 필드는 'AR vs diffusion'을 서로 다른 종이 아니라 하나의 schedule의 양 끝점으로 다루는 decoder를 향해 가는 듯하다.
Sparks — where one could take this 아이디어의 씨앗
-
One could systematically map where dLLM decoding loses to compute-bound AR serving across sequence length, prefill, and batch size — extending the batch-scaling observation in the papers own appendix into a full measurement. Sequence 길이·prefill·batch size에 걸쳐 dLLM decoding이 compute-bound AR serving에 밀리는 지점을 체계적으로 지도화해볼 수 있다 — 논문 부록의 batch-scaling 관찰을 정식 측정으로 확장하는 것이다.
-
One could characterize why multimodal decoding (the >8% LLaDA-V swing) is so much more block-size sensitive than text — a gap the paper reports but does not explain. 왜 multimodal decoding(LLaDA-V의 >8% 스윙)이 텍스트보다 block size에 훨씬 민감한지 특성화해볼 수 있다 — 논문이 보고만 하고 설명하지 않은 격차다.
-
One could extend the greedy-equivalence result of Theorem 1 toward sampling-based decoding, deriving confidence thresholds that provably bound the distance to the true joint under temperature sampling. Theorem 1의 greedy-equivalence 결과를 sampling 기반 decoding으로 확장해, temperature sampling에서도 true joint와의 거리를 증명 가능하게 bound하는 confidence threshold를 유도해볼 수 있다.
Seeds grounded in the paper's own stated limitations and open questions. 논문이 스스로 밝힌 limitations와 open questions에 뿌리를 둔 씨앗입니다.
Notes
Notes
Fast-dLLM is the paper that made open diffusion-LLM inference feel competitive: it identifies the two structural reasons LLaDA and Dream decode slowly — no KV cache under bidirectional attention, and dependency violations under naive parallel commits — and fixes both without any retraining. The block-wise approximate cache turns “how stale can cached states get?” into a tunable block-size knob, while the (n+1)·ε ≤ 1 condition of Theorem 1 gives the confidence-threshold heuristic an actual correctness footing rather than leaving it purely empirical. Its main honestly-stated boundary is batching: at larger batch sizes the full-attention compute floor lets autoregressive models pull back ahead, which is exactly where follow-up work on dLLM serving now concentrates.
LLaDA 계열 dLLM 가속 논문을 읽을 때 사실상의 공통 베이스라인이므로, block size와 confidence threshold 두 knob의 trade-off 곡선을 기준점으로 잡고 후속 논문들을 비교하면 좋다.
Linked reviews
LLaDA: Large Language Diffusion Models
An 8B masked-diffusion language model trained from scratch on 2.3T tokens that matches LLaMA3-8B-class autoregressive baselines, showing next-token prediction is not the only recipe for LLM capabilities.
quantizationKIVI: A Tuning-Free Asymmetric 2-bit Quantization for KV Cache
Asymmetric KV quantization derived from element-distribution analysis — per-channel keys, per-token values — pushes the KV cache to tuning-free 2 bits, cutting peak memory 2.6x and lifting serving throughput up to 3.47x.
speculative-decodingAdaFlash: Adaptive Speculative Decoding via On-Policy Distilled Diffusion Drafters
AdaFlash stabilizes diffusion-model drafters for speculative decoding, whose bidirectional attention otherwise causes high acceptance-rate variance across domains and token positions, via on-policy reverse-KL distillation plus an adaptive candidate-length head, reaching up to about 66% higher throughput in high-concurrency serving.
diffusion-llmAdaLook: Adaptive Multi-Step Lookahead Decoding for Diffusion Language Models
AdaLook makes diffusion-LM lookahead decoding adaptive-depth instead of fixed-depth — a variance-gated continue/stop rule plus branch expansion from informative intermediate states — Pareto-dominating one-step lookahead and confidence-only decoding on the accuracy/decoding-steps trade-off across MMLU, GSM8K, MATH500 and BBH.
diffusion-llmCARVE: Counterfactual-Aware Reveal with Verified Expansion for Diffusion Language Models
CARVE lets a masked diffusion language model grow its answer canvas mid-generation, accepting the extra space only when a counterfactual stability check confirms the insertion will not disturb the model's existing predictions.
dllmConstrained Decoding for Diffusion Language Models via Efficient Inference over Finite Automata
An exact algorithm samples from the constraint-satisfying mean-field posterior of a diffusion language model for any constraint expressible as a finite automaton, fixing a correctness gap left open by autoregressive-style token masking and lifting Dream-7B's BFCL-Live function-calling accuracy under stochastic sampling from 22.3% to 69.0% with under 5% overhead.
dllmFlowBlock: Wavefront-Parallel Decoding for Self-Correcting Diffusion Language Models
FlowBlock is a training-free decoding framework for block-diffusion language models that turns each block's self-correction ability into scheduling slack, letting adjacent blocks decode in an overlapping wavefront instead of strictly one-after-another, for up to 4.01x higher throughput and a small accuracy gain over serial LLaDA-2.x decoding.
dllmLaCache: Exact Caching and Precision-Adaptive Inference for Diffusion Large Language Models
LaCache is a training-free acceleration framework for diffusion LLMs that eliminates redundant denoising-step computation via exact intermediate-result caching and adds per-group FP8 quantization to FFN layers, reaching 1.3x speedup alone and up to 40.2x when stacked with existing acceleration methods.
diffusion-llmMulti-Mask Diffusion Language Models for Few-Step Generation
MultiMDM fixes a structural obstacle to few-step generation in masked diffusion language models -- all forward trajectories collapsing to the same single fully-masked state, leaving no terminal entropy for consistency distillation -- by replacing the single mask token with a designated set of masks per clean token, preserving masking structure while restoring terminal diversity.
dllmNemotron-Labs-Diffusion: A Tri-Mode Language Model Unifying Autoregressive, Diffusion, and Self-Speculation Decoding
A single NVIDIA model switches between autoregressive, block-wise diffusion, and self-speculative decoding from one set of weights, with self-speculation (diffusion drafts, the same model's AR pass verifies) beating multi-token prediction and the 8B model decoding 6x more tokens per forward than Qwen3-8B at comparable accuracy.
dllmSangam: Efficiently Serving Diffusion LLMs with the AR Stack
A dedicated serving system for cached diffusion-LLM inference introduces a deficit token-budget scheduler that manages prefill/decode interference without AR-style chunked prefill, sustaining 3x the query rate of an in-system Fast-dLLM baseline on LLaDA-8B and Dream-7B.
diffusionSeeing the End at Step Zero: Accelerating Diffusion MLLMs via MLP Sparsity-Aware Truncation
Seer exploits a sharp MLP activation-sparsity jump at diffusion step zero to predict a diffusion multimodal LLM's true output length before generation, then permanently truncates the redundant end-of-text padding and routes batches through a hybrid static/variable-length execution scheme, reaching up to 31x throughput gains across nine vision-language benchmarks without hurting accuracy.
diffusion-llmServing Masked Diffusion LLMs: Characterization and Design Principles from Real Hardware
The first real-hardware characterization of masked-diffusion LLM serving finds that denoising-step difficulty clusters into 11 discrete, unpredictable levels, that CPU dispatch (not GPU compute) is the dominant per-request cost, and derives a batch-timeout rule from measured queueing behavior.
diffusion-llmSurvival-Guided Length Control for Efficient Diffusion Language Models
Recasts how long a diffusion language model should generate as a discrete-time survival problem over the end-of-sequence token, yielding a training-free plug-in that speeds up decoding 3-7x with no accuracy loss.