Research

Nonequilibrium physics of neural computation

We are interested in the nonequilibrium dynamics of neural systems, which operate far from equilibrium and exhibiting time-asymmetric, irreversible processese. Unlike classical neuroscience theories that focus on stationary, equilibrium information paradigms (e.g., efficient coding theory, associative memories), we study how neural computation involves dynamic, non-stationary processes. Using the nonequilibrium statistical physics of asymmetric, disordered Ising models as a prototypical framework, we aim to develop an exact analytical theory to understand the thermodynamic properties and phase transitions of large-scale nonequilibrium networks. This approach sheds light on the role of nonequilibrium fluctuations in effective neural information processing and their interaction with sensorimotor streams and internal states.

• Aguilera M, Igarashi, M & Shimazaki, H (2023). Nonequilibrium thermodynamics of the asymmetric Sherrington-Kirkpatrick model. Nature Communications 14, 3685.
  

Statistical Physics of Transformer Neural networks

We exploit methods from nonequilibrium statistical physics and the disordered physics of associative memories to describe the macroscopic variables that guide the behavior of attention mechanisms in transformer models. By investigating the relationship between attention mechanisms and hetero-associative memories (aka asymmetric Hopfield networks), we aim to uncover the complex dynamics of such networks. Using path integral methods over generating functionals, we derive analytical approximations for large self-attention neural networks, revealing phenomena such as nonequilibrium phase transitions and chaotic bifurcations. With this, we aim to develom methods to enhance the interpretability of transformer models and to reduce computational training costs.

• Poc-López A & Aguilera M (2024). Dynamical Mean-Field Theory of Self-Attention Neural Networks. arXiv preprint.
  

Stochastic Thermodynamics of Autonomous Agents

We are studiyng how to use stochastic thermodynamics to formalize the capacities of autonomous agents, particularly their ability to maintain far-from-equilibrium states by actively harvesting and storing free energy from their environments. By studying nonequilibrium steady states, we identify conditions under which agents can sustain themselves autonomously, differentiating these from states near thermodynamic equilibrium with low energy-harvesting potential. Additionally, we explore the adaptive mechanisms that allow agents to regulate their interactions with the environment, ensuring the robust maintenance of their nonequilibrium states. This research line aims to deepen our understanding of how thermodynamic processes underpin the emergence and persistence of complex, adaptive behavior in natural and artificial systems.