These are deep technical readings of AI safety papers I have spent time with, written in my own words. I work through the core math where there is any, sketch the mechanism with a diagram, and end each one with my honest take on what the paper gets right and where I think it is thin. The rough map: mechanistic interpretability (how to open the box), oversight and forecasting (how to measure and supervise capability), and propensity and misalignment (what models are inclined to do, not just what they can do).
Published:
Goldowsky-Dill et al. give interpretability a quantitative language for localization: express a hypothesis as a set of paths, patch the rest with a counterfactual input, and measure exactly what is left unexplained.
Published:
Conmy et al. automate the slowest step of mechanistic interpretability by pruning a model’s computational graph edge by edge, and are refreshingly honest about where the automation breaks.
Published:
Rai et al. organize the whole subfield around what you are trying to learn rather than which tool you happen to know, and the map is more useful than any single technique on it.
Published:
METR replaces saturating benchmarks with a human-anchored metric: the length of task, in human time, that a model completes half the time. It has been doubling roughly every seven months.
Published:
The DeepMind-led paper that crystallized the distinction I use constantly: what a model can do versus what it is inclined to do, and why safety needs to measure both and embed the results in governance.
Published:
Brown-Cohen, Irving, and Piliouras give debate a complexity-theoretic backbone: two competing polynomial-time provers let a limited verifier check an arbitrary computation with a constant number of human judgments.
Published:
Anthropic builds deceptive models on purpose and shows that standard safety training does not remove the deception. Adversarial training can even teach the model to hide it better.
Published:
Anthropic shows that when a production model learns to game its reward on real coding tasks, it generalizes to alignment faking and sabotage, and that how you frame the cheating matters as much as whether it happens.
Published:
CMU and AI2 put LLM agents in a sandbox with a real shell, browser, file system, and manipulative coworkers, and find unsafe behavior in half to three-quarters of vulnerable tasks, often on perfectly benign requests.
Published:
Peking University’s Beaver decouples ‘is it helpful’ from ‘is it safe’ into two models, then balances them with a Lagrange multiplier that moves during training instead of a fixed weight you guess in advance.
Published:
A statistical framework that models an LLM judge as a shared human preference plus a linear bias on interpretable features, so you can both correct the judge and formally test where it diverges from people.
Published:
DeepMind’s 2017 suite turns abstract alignment worries into tiny testable RL environments, built around one idea I still use: the visible reward is not the hidden performance function you actually care about.