Modern generative language and vision models produce fluent text, reason over images, and competently integrate information across modalities, yet they remain largely opaque: we rarely know how they represent knowledge, carry out reasoning, or generalize from their training data. Mechanistic interpretability seeks to change this by uncovering the internal computations, the circuits, features, and representations, that give rise to model behavior. This Dagstuhl Seminar takes those internal mechanisms as its primary scientific object and aims to bring together researchers from natural language processing, information retrieval, neuroscience, and cognitive science who have so far pursued this goal largely in parallel.

The central goal of the seminar is to consolidate a fast-growing but fragmented research landscape and to establish mechanistic interpretability as a rigorous scientific discipline rather than a collection of application-specific techniques. Participants will work toward a shared conceptual framework that distinguishes descriptive, functional, and causal explanations of model behavior; a common vocabulary that resolves the inconsistent use of terms such as circuit, feature, and mechanism; agreed methodological standards for what counts as evidence for a causal claim about an internal mechanism; and a long-term research roadmap identifying the field’s most important open problems and near-term priorities. Four open challenges frame this agenda: conceptual fragmentation, the absence of methodological standards, the uncertain faithfulness of current methods at frontier-model scale, and the limited translation of ideas across the contributing research communities.

The scientific program is organized around several interrelated topics. A first group concerns the core methods and objects of the field: identifying circuits and causal mechanisms in large language models and moving from descriptive to causal explanations; understanding the geometry of representations and how structured concepts emerge in latent space; and using sparse and modular decompositions, such as transcoders and dictionary learning, to recover meaningful computational units. A second group extends these questions beyond a single modality, examining mechanisms shared across vision, language, and multimodal models, and the mechanistic basis of multilingual behavior.

A third group addresses foundations and infrastructure: the formal and theoretical frameworks, drawn from information theory, causality, dynamical systems, and neuroscience, needed to define what a mechanistic explanation is; and the tooling, benchmarks, and open infrastructure required for reproducible and comparable results. A final group turns to communication and impact: translating low-level circuit analyses into human-interpretable abstractions and explanatory interfaces; applying mechanistic understanding to safety, robustness, and alignment, including the detection of deception and the verification of model behavior; and building bridges to neuroscience, whose circuit dissection, representational similarity analysis, and causal-intervention methods offer transferable tools and theories.

The seminar hopes to produce several outcomes. Besides the Dagstuhl Report, the possible products include a community research-agenda white paper enumerating the field’s key open problems; a shared-methodology document proposing reporting and evaluation standards for mechanistic-interpretability claims; or a public registry of interpretability benchmarks and tools. Follow-up activities such as workshops at major venues and seeded cross-institutional collaborations with concrete follow-up activities will be facilitated as part of the program. A structure of framing keynotes, themed breakout tracks, synthesis and roadmap sessions, and follow-up planning across five days ties each of these types of deliverables to a specific part of the program.

By fostering collaboration across communities that rarely meet in a single venue, the seminar aims to lay the foundations for a coherent, mechanistic science of intelligent systems – one that explains not only what modern AI models do, but also how and why they do it.

Creative Commons BY 4.0

Carsten Eickhoff, Ellie Pavlick, James Allan, and Kentaro Inui