TopoLM

Abstract

Neurons in the brain are spatially organized such that neighbors on tissue often exhibit similar response profiles. In the human language system, experimental studies have observed clusters for syntactic and semantic categories, but the mechanisms underlying this functional organization remain unclear. Here, building on work from the vision literature, we develop TopoLM, a transformer language model with an explicit two-dimensional spatial representation of model units. By combining a next-token prediction objective with a spatial smoothness loss, representations in this model assemble into clusters that correspond to semantically interpretable groupings of text and closely match the functional organization in the brain's language system. TopoLM successfully predicts the emergence of the spatio-functional organization of a cortical language system as well as the organization of functional clusters selective for fine-grained linguistic features empirically observed in the human cortex. Our results suggest that the functional organization of the human language system is driven by a unified spatial objective, and provide a functionally and spatially aligned model of language processing in the brain.

Introduction

The human brain exhibits spatio-functional organization: across the cortical sheet neurons with similar response properties are often spatially clustered (Hubel & Wiesel, 1962, Kanwisher 1997, Tootell 1998, Kaschube 2010). This principle underlies the structure of many systems, including the visual cortex and the human language system—a network of left-lateralized frontal and temporal regions selectively engaged in language processing (Fedorenko et al., 2010; Fedorenko et al., 2024). Despite growing evidence that transformer-based language models align well with brain responses (Schrimpf et al., 2021; Caucheteux & King, 2022), these models lack an explicit spatial dimension. TopoLM addresses this gap by embedding model units in a two-dimensional grid and training with a spatial smoothness loss (Lee 2020, Margalit, 2024). Without ever training on neural data, topographic organization emerges in TopoLM, leading to clusters selective for linguistic features and resembling the spatial layout of the human language system. TopoLM belongs to the Topographic Deep Artificial Neural Network (TDANN) family of models (Lee et al., 2020; Margalit et al., 2024). Margalit et al. (2024) originally posited the loss term underlying TDANN models as a unifying account of the develop-ment of spatio-functional organization in the visual cortex. TopoLM successfully extends the corresponding spatial loss to the domain of language neuroscience, and thus provides evidence that this principle of spatial smoothness indeed generalizes across cortex.

Experiments & Findings

Using 3 independent neuroimaging datasets (Fedorenko 2024, Hauptman 2024, Moseley 2014) we show that TopoLM predicts key linguistic clusters in the human cortex.

Finding I: TopoLM exhibits a core language system similar to the core human language system (Fedorenko 2010, 2024).

Finding II: TopoLM predicts clusters selective to linguistic categories such as verbs and nouns (Hauptman 2024) or concrete verbs and concrete nouns (Moseley 2014)

Finding III: The spatial loss comes at virtually no cost with regard to downstream performances or brain alignment.

BibTeX


          @inproceedings{rathi_topolm_2025,
            title = {TopoLM: brain-like spatio-functional organization in a topographic language model},
            url = {http://topolm.epfl.ch/},
            doi = {10.48550/arXiv.2410.11516},
            language = {en},
            booktitle = {International {Conference} on {Learning} {Representations}},
            author = {Rathi, Neil and Mehrer, Johannes and AlKhamissi, Badr and Binhuraib, Taha and Blauch, Nicholas M and Schrimpf, Martin},
            year = {2025}
          }

Acknowledgement

This website is adapted from The LLM Language Network, LLaVA-VL, Nerfies, and VL-RewardBench, licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.