Technical Writings
Deep dives into technical topics, tutorials, and explorations in software engineering and machine learning.
- A Contrarian Take on the Alignment Problem
Why aligning superintelligence to 'human values' might be aiming at the wrong target, and what it could mean to build the values in before the knowledge.
- The Alignment Problem and Shoggoth Meme
First in the series about understanding AI misalignment. Meme are getting real!!
- Self-Evolving Search Agents: How LLMs Learn Without Training Data
Exploring Dr. Zero's framework where LLM agents bootstrap their own training data through self-play, enabling continuous improvement without human annotation.
- Learning to Adapt in Test-Time (Titans/MIRAS)
A deep dive into Titans and MIRAS architectures that enable LLMs to memorize and adapt at inference time using neural memory modules.
- Towards Infinite Context: How LLMs Are Breaking the Context Limit
A comprehensive guide to extending LLM context windows through position encodings, efficient attention, and memory augmented architectures.
- Attention That You Probably Didnt Know Existed!!
From sparse patterns to linear attention and state space models exploring the zoo of efficient attention mechanisms that go beyond vanilla transformers.