Neurotheory
While neuroscientists have increasingly powerful deep learning models that predict neural responses, it is not clear that these models are correspondingly increasing our understanding of what neurons are actually doing. In this session, we will take a more mechanistic approach to understanding how networks of neurons afford complex computations, both by both considering mechanistic neural model along with mathematical theories that say how neurons should behave and crucially why they behave that way.
Session Chair
Dr James Whittington (University of Oxford; Stanford University)
A unifying framework for frontal and temporal representation of memory
Invited Talks
Dr Thomas Parr (University of Oxford)
From models to maladies
Contributed Talks
Dr Tommaso Salvatori (Verses.ai)
On the past, present, and future of predictive coding
Carol Upchurch (Louisiana State University)
Persistent Silencing of PV+ Inhibitory Interneurons Results from Proximity to a Subcritical Hopf Bifurcation
Tyler Giallanza (Princeton University)
Adapting to a changing environment with controlled retrieval of episodic memories
Declan Campbell (Princeton University)
Unraveling geometric reasoning: A neural network model of regularity biases
Invited Talks
Dr Thomas Parr
University of Oxford
This talk aims to do two things. The first is to overview the idea that our brains—and perhaps other sentient systems—employ internal models of our surroundings to interpret and act upon the world. The second is to consider what happens when we employ suboptimal models that are a poor fit to our sensory data. The imperative to fit our models to our data in health—and the cost of deviating from this fit in disease—is central to a theoretical perspective known as Active Inference. This approach is based upon the principle that action and perception both serve to optimise our fit to our world. During this talk, we will discuss the ways in which this principle can be used to aid our understanding of brain connectivity, behaviour, and computational pathologies.
From models to maladies
Contributed Talks
Verses AI
On the past, present, and future of predictive coding
This talk will discuss past, present, and future of predictive coding, with a special emphasis on potential applications in machine intelligence. To do this, we will first summarize the journey that has brought predictive coding from being considered a compression algorithm for time series data, to its contemporary role as an evidence-maximization framework for a specific type of generative models. Then, we will provide an informal definition of what predictive coding is today, and use it as an umbrella for different algorithms that fall under this definition.
We will then contextualize where predictive coding stands in a community where the vast majority of the research is focused on the improving of backpropagation-based models. To do so, we will survey theoretical results that show differences and similarities among the two algorithms. A special focus is then placed on the strong points of predictive coding, such as its natural robustness, flexibility, and biological plausibility. Predictive coding indeed shows advantages on problems that are faced by biological organisms, such as continual learning, online learning, and learning from a small amount of data. Furthermore, the bio-plausibility of its update rules is particularly suited for implementations on neuromorphic chips.
So, where is predictive coding headed? The conclusive part of this talk will be focused in providing an answer to this question, and will cover a list of the challenges that need to be met to allow prevent predictive coding to be used in practical tasks.
Carol Upchurch
Louisiana State University
Persistent Silencing of PV+ Inhibitory Interneurons Results from Proximity to a Subcritical Hopf Bifurcation
Persistent activity in excitatory principal cells has been suggested as a mechanism to maintain memory traces during working memory. Persistent interruption of firing in PV+ inhibitory interneurons has been demonstrated and suggested to be the inhibitory counterpart of persistent activity in excitatory neurons. Since an interruption in inhibitory interneuronal firing disinhibits the excitatory principal cells, it could serve a similar function. We find the mechanism is similar in models of neocortical and hippocampal area CA1 fast spiking interneurons. An inhibitory synaptic potential or hyperpolarizing current that decays sufficiently slowly is applied to a repetitively firing interneuron. This stimulus removes inactivation from rapidly activating, slowly inactivating potassium current KV1, bringing the membrane potential to a temporary equilibrium above the spike threshold during repetitive firing. After the stimulus has faded, KV1 continues to de-inactivate slowly until KV1 is sufficiently weak that repetitive firing can be reestablished.
A reduced model using the inactivation variable as a bifurcation parameter shows the initial stimulus collapses the limit cycle for repetitive firing onto a co-existing stable fixed point corresponding to depolarization block. As this variable decays, the trajectory crosses a supercritical Hopf bifurcation following the curve of unstable fixed points until it spirals out into repetitive firing. In an alternative model describing entorhinal cortical PV+ interneurons, similar behavior is achieved without a slowly activating/inactivating gated current. When the baseline current is close enough to a Hopf bifurcation, the fixed point is weakly repelling, taking hundreds of milliseconds to resume firing after an interruption.
Princeton University
Adapting to a changing environment with controlled retrieval of episodic memories
The flexibility of human cognition has long been associated with both episodic memory and cognitive control. To investigate the interactions between these systems, we introduce the Episodic Generalization and Optimization (EGO) framework. In this framework, EM rapidly learns relationships among states while cognitive control coordinates memory encoding and retrieval by monitoring current task demands and maintaining a representation of the current task context. The combined system makes decisions in novel environments by matching both the current environmental state and the current control state to its memories. The system can also plan and make sequential decisions by iteratively activating its memories (“preplay”). We use the framework to simulate behavioral data from two previously unconnected tasks — revaluation during sequential decision making and multi-task learning with aliased states — that both require online adaptation to a changing environment. The framework provides a coherent account of the biases present in the behavioral data, such as a preference for reward revaluation over transition revaluation and a preference for blocked curricula in multi-task learning. The results suggest that EM’s instance-based nature facilitates rapid learning, even when faced with long-term, non-Markovian dependencies between states, but at the cost of interference between related memories. We show that the control system helps navigate this tradeoff by manipulating the similarity between memories, reducing interference by learning task-specific control representations that help retrieve only the task-relevant memories. Overall, the EGO framework provides a biologically plausible process model of model-based decision making while highlighting the role of cognitive control in the real-time optimization of performance.
Spotlight Talks
NYU
Coherence influences the dimensionality of communication subspaces
The brain relies on communication between specialized cortical areas to accomplish complex cognitive tasks. There are two leading hypotheses for communication between cortical areas: 1) The communication through coherence (CTC) hypothesis posits that coherent oscillations are required for information propagation. 2) The information transmission via communication subspace (CS) hypothesis, advances the idea that low-dimensional subspaces of population activity are responsible for communication across cortical areas. There is a clear divide between these two mechanisms and the CTC hypothesis, in particular, CTC has been surrounded by considerable skepticism, with many authors reducing oscillations in the cortex to an epiphenomenon. Here, we reconcile these two mechanisms of communication through a spectral decomposition of communication subspaces. In our main result, we predict that coherence influences the dimensionality of the CS: Dimensionality is lowest, and the prediction performance is highest, at frequencies that exhibit a peak in coherence between a source and target population. We arrive at these results by developing an analytical theory of communication for circuits described by stochastic dynamical systems exhibiting fixed-point solutions. We compute directly the predictive performance for the mean-subtracted activity of a target population from that of a source population and show that our predictions agree with the experimental results. Then via a band-pass filtered version of the covariance matrix, we arrive at our main result. Hence, our theory makes experimentally testable predictions of how oscillations influence interareal communication while advancing a new hypothesis for the functional role of oscillatory activity in the brain.
Declan Campbell
Princeton University
Unraveling geometric reasoning: A neural network model of regularity biases
Uniquely among primates, humans possess a remarkable capacity to manipulate abstract structure in service of task goals across a broad range of behaviors. Previous studies have demonstrated that this sensitivity to abstract structure extends to the visual perception of geometric forms. These studies have highlighted a uniquely human bias toward geometric regularity whereby task performance for more regular and symmetric forms is enhanced compared to their geometrically irregular counterparts. Computational models investigating geometric reasoning have revealed that a wide array of neural network architectures fail to replicate the performance biases and generalization exhibited by human behavior. In this study, we propose a neural network architecture augmented with a simple relational inductive prior. When trained with the appropriate curriculum, this model demonstrates similar biases towards symmetry and regularity in two distinct tasks involving abstract geometric reasoning. Our findings indicate that neural networks, when equipped with the necessary training objectives and architectural elements, can replicate human-like regularity biases and generalization. This approach provides valuable insights into the neural mechanisms underlying geometric reasoning and offers a promising alternative to prevailing symbolic language of thought models in this domain.