✎ thoughts and notes on AI/ML research & software engineering
No one read the blogs, but I kept writing anyway.
At the start of David Wells’s book The Penguin Dictionary of Curious and Interesting Numbers, there is a list of 98 mathematicians in order of time, from Ahmes in ancient Egypt to Srinivasa Ramanujan in India (Wells, 1987, pp. 9–11). In this post, I turn that list into charts, so we can see when they lived, where th...
Systems that use more than one LLM already exist. One model writes code. Another does math. Another is a general assistant. They are asked to work together. Today they talk in English. The first mo...
The term mechanistic interpretability now appears in workshops, reading groups, grants, and much of the new work on language models. The growth has not come with a shared definition. Researchers, w...
Every GPT-style language model is trained with the same objective. The transformer takes a context of tokens, the last layer maps that representation onto the vocabulary, and a softmax turns the sc...
AI coding agents are no longer a novelty. They are being used in production workflows at companies of every size, praised in engineering blogs, featured in product launches, and adopted by develope...