Singular learning theory (SLT) identifies the geometry of the loss landscape as key to understanding neural networks. In this talk, I will explore applications of this framework and perspective for interpretability, alignment, and other areas of AI safety.

Jesse Hoogland and Daniel Murfet, founders of Timaeus, introduce their mathematically rigorous approach to AI safety through 'developmental interpretability' based on Singular Learning Theory.

You may have heard of singular learning theory, and its 'local learning coefficient', or LLC - but have you heard of the refined LLC? In this episode, I chat with Jesse Hoogland about his work on SLT, and using the refined LLC to find a new circuit in language models.

Singular Learning Theory (SLT) is a novel mathematical framework that expands and improves upon traditional Statistical Learning theory using techniques from algebraic geometry, bayesian statistics, and statistical physics. It has great promise for the mathematical foundations of modern machine learning.

Singular learning theory (SLT) suggests a correspondence between: structure of the data; internal algorithmic structure of a neural network; geometry of the loss landscape; and structure of the learning process. This provides a basis for developing tools & theory to interpret what neural networks learn and how they learn it. In this talk, we present an introduction to SLT, survey recent evidence for different parts of this correspondence, and sketch future directions and applications within AI safety & interpretability.

We show that in-context learning emerges in transformers in discrete developmental stages, when they are trained on either language modeling or linear regression tasks. We introduce two methods for detecting the milestones that separate these stages, by probing the geometry of the population loss in both parameter space and function space. We study the stages revealed by these new methods using a range of behavioral and structural metrics to establish their validity.

by Dan Murfet and Jesse Hoogland. A big thanks from the entire organising team to our sponsors and to the Topos Institute for making this all possible. Without your support, this summit could not have taken place!

The state of AI safety (as of June 2023).

Singularities and nonlinear dynamics (following e.g. Strogatz). By Jesse Hoogland.

Statistical mechanics, Boltzmann distribution, free energy, phases and phase transitions. By Jesse Hoogland.

Jesse Hoogland is a research assistant at David Krueger's lab in Cambridge studying AI Safety. More recently, Jesse has been thinking about Singular Learning Theory and Developmental Interpretability, which we discuss in this episode.

What can we learn about neural networks’ internals by looking at the loss landscape? Quite a lot, or so Singular Learning Theory (SLT) tells us. Just as the geometry of the energy landscape determines a system’s physical properties, the structure of the loss landscape determines many of a model’s computational properties. In this talk, we explore what SLT has to tell us about interpreting neural networks and detecting phase transitions during training.