✎ thoughts and notes on AI/ML research & software engineering
No one read the blogs, but I kept writing anyway.
Language models now talk about their inner lives (Anthropic, 2025b; Berg, de Lucena and Rosenblatt, 2025). Ask one how it feels, and it will often answer in fluent, careful, first-person prose. In 2022, a Google engineer said that the LaMDA chatbot was sentient, and Google replied that “there was no evidence that La...
At the start of David Wells’s book The Penguin Dictionary of Curious and Interesting Numbers, there is a list of 98 mathematicians in order of time, from Ahmes in ancient Egypt to Srinivasa Ramanuj...
Systems that use more than one LLM already exist. One model writes code. Another does math. Another is a general assistant. They are asked to work together. Today they talk in English. The first mo...
The term mechanistic interpretability now appears in workshops, reading groups, grants, and much of the new work on language models. The growth has not come with a shared definition. Researchers, w...
Every GPT-style language model is trained with the same objective. The transformer takes a context of tokens, the last layer maps that representation onto the vocabulary, and a softmax turns the sc...