Writings
A collection of my literary atrocities on my interests, passions, and experiences.
Understanding Multi-Head Attention Mechanism
September 6, 2026
Understanding Multi-Head Attention Mechanism
To write a CUDA Matmul Kernel That Reaches 97% of cuBLAS Performance
August 10, 2026
Progressively optimizing a CUDA matmul kernel from a naive implementation to 97% of cuBLAS performance, one bottleneck at a time.
From CUDA Hardware Architecture To CUDA Kernels
August 9, 2026
The GPU hardware, the CUDA programming model, and the execution model that maps one onto the other. The primer behind the optimized GEMM series.
Global Memory Coalesing From DRAM Up
July 31, 2026
DRAM row mechanics, 32-byte sectors, and the 13x you lose to uncoalesced access.
A Small cuBLAS Detail With a Deep Lesson
July 23, 2026
The reasoning behind cuBLAS's operand swap, and the layout lesson hiding inside it.
Why almost everyone has poor posture
June 21, 2026
Understanding the fundamental principles of biomechanics and a practical guide on fixing your posture