In-depth engineering notes, deep dives, and architectural explorations.
A deep dive into Key-Value caching mechanics, memory fragmentation bottlenecks, PagedAttention, Multi-Query Attention (MQA), and speculative prefix caching.
How memory-mapped buffers, cache line alignment, and Apache Arrow columnar formats eliminate serialization overhead in high-throughput analytical streaming.
Why vibe-checking fails in production: designing synthetic datasets, calibrated LLM-as-a-judge rubrics, trajectory state checking, and statistical significance testing.
An analysis of transactional outbox patterns, dual-write pitfalls, idempotent consumers, and two-phase commits vs saga orchestrations.