Survey explores how understanding AI could help make it safer

Senior Researcher Dr Fazl Barez, DPhil Student Tung-Yu (Tony) Wu and a team of international collaborators have published a comprehensive survey of mechanistic interpretability – a growing area of AI research that aims to understand how AI models work from the inside.

Dr Barez leads the Technical Safety and Governance (TSG) lab in Information Engineering at Oxford, focusing on understanding the internal mechanisms of neural networks and developing methods to make advanced AI systems more transparent, reliable, and controllable.

As AI systems become increasingly powerful and widely used, understanding how they reach their decisions is an important part of efforts to ensure they behave safely. But does knowing how an AI model works necessarily make it safer?

This is one of the central questions explored in the survey, which examines the different approaches researchers are taking and how they might contribute to AI safety.

Mechanistic interpretability involves looking inside AI models to understand how information is processed and how particular behaviours or decisions emerge. Researchers use a range of techniques to identify the internal components and processes responsible for a model's outputs, and to investigate how these components work together.

The field has developed rapidly, bringing together researchers with different methods and ideas about what it means to understand an AI model. The survey maps out these approaches, covering techniques for identifying important features within models, tracing how different components work together, testing whether proposed explanations are correct, and understanding how models learn.

An important challenge is knowing whether an explanation of an AI model's behaviour is actually accurate and useful. The researchers propose a framework for evaluating explanations across five areas: how accurately they identify the processes behind a model's behaviour; how completely they explain it; whether the findings remain consistent; whether they can be verified through testing; and how easily people can understand them.

The authors emphasise that understanding how AI models work is not, on its own, a solution to the challenges of AI safety. However, interpretability could provide valuable tools for identifying hidden or unexpected behaviours, influencing how models represent information, correcting what they have learned, and developing automated systems to help researchers investigate AI models.

Alongside the paper, the team has developed an interactive online resource to help researchers and students explore the field and investigate the techniques themselves.

The work will contribute to an upcoming handbook on mechanistic interpretability for AI safety. It has already informed teaching at the University of Edinburgh and will provide the foundation for Fazl's course, Interpretability for AI Safety, at the University of Oxford in Hilary term 2027.

Explore the research

Read the survey paper

Explore the interactive learning resource