Running a 20B Language Model at 218 Tokens/Second on a Mac Mini
August 4, 2026 · 22 min read · on DeepGrove
I led inference on this. 218 tok/s decode and 1,075 tok/s prefill on a $599 16 GB Mac Mini, 5–16× faster than Gemma 4, Qwen3.5, and GPT-OSS.
writing
Making Reactive-Diffusion Systems Faster, A Deep Dive into Kernel Optimization with Triton
July 20, 2025 · 15 min read
Optimizing Gray-Scott reaction-diffusion simulations using Triton GPU kernels, achieving a 20x speedup through memory optimization and kernel fusion techniques.
Inferencing Transformers in Real-Time
May 13, 2025 · 20 min read · on Substack
A journal on writing CUDA kernels from scratch to run GPT-2 at almost 70 tokens per second on an A40 GPU, exploring optimization techniques from tensor cores to flash attention.