Research
The cost of inference is bytes moved,
not math done.
Modern language-model inference rarely runs out of arithmetic — it runs out of memory bandwidth. Weights and KV caches stream through HBM far slower than tensor cores can consume them, so the useful question is almost never "how do we compute faster?" but "how do we move less?" I study training-free ways to answer that for diffusion LLMs and their neighbors: the interesting problems live at the boundary of algorithms and hardware, where a decoding rule, a cache layout, and a roofline plot all constrain each other.
Research areas
Four rooms, one wall
Different model families, same recurring bottleneck: memory movement. Each area below is a different angle on that one wall.
Diffusion LLM inference
Diffusion LLMs decode many tokens per step instead of one, which rewrites the serving economics we take for granted in autoregressive models. I work on parallel decoding schedules, what can safely be cached across denoising steps, and measuring dLLM inference by end-to-end serving cost and quality — not per-step proxies.
KV-cache compression & quantization
At long context and large batch, the KV cache — not the weights — is what caps how many requests fit on a GPU. I study low-bit and structured representations of keys and values that shrink that footprint while keeping accuracy, and why keys and values deserve different treatment in the first place.
Speculative & parallel decoding
Speculation trades cheap drafts for expensive verifications, and the whole bargain hinges on acceptance behavior. I care about the latency structure underneath: when drafting pays for itself, how acceptance rates move with the task, and how these ideas compose with parallel decoders rather than compete with them.
Video & multimodal diffusion efficiency
Video diffusion transformers hit the same memory-wall story with the volume turned up: enormous attention states, heavy redundancy across denoising steps and frames. The open question is which text-side lessons transfer — and which quietly fail — when the tokens are spatiotemporal.
How I work
Three habits I try not to break
Observe before you optimize.
A bottleneck is an empirical fact, not an assumption. Before touching any technique, profile where the bytes and the milliseconds actually go — the measured system is allowed to disagree with the paper diagram, and usually does.
A method must be forced by a mechanism.
A good method is the corollary of an explanation: first understand why the cost arises, then let the fix follow from that understanding. If a simpler baseline would work, the honest move is to use it and say so.
Report what the measurement says.
Matched baselines, controlled comparisons, and numbers taken from runs that actually finished. Negative results are results — an experiment that rules out a plausible story earns its keep.
Publications & preprints
Selected work
First entries are in the pipeline.
Until they land here, the writing lives elsewhere: I keep a running set of reviews of other people's papers — dLLMs, KV caching, sparse attention — on the Paper Review page.
Read the paper reviews