Internal models of Songbird

Abstract

The notion that the brain is an intricately organized web of a neural network essentially performing feature detection in a hierarchical manner, seems to a back seat. A novel view first presented by Helmholtz in the 19th century, suggests that the brain forms a generative model, and hypothesizes the causes of the events taking place which are eventually tested against incoming sensory evidence, updating its neural representation to form a perception. Thus, the brain can be thought of as a Bayesian Inference Machine. Predictive coding theory is the most popular framework under the family of the Bayesian brain models. This theory posits the brain encodes not the features of sensory stimuli, but rather the statistical irregularities present in it. There is substantial evidence to support this theory, mostly found in human or rodent studies. However, the songbirds have the potential to serve as an excellent model organism to be studied in the light of this framework since the birdsong is a fine example of hierarchical sensorimotor information processing. This review intends to give a primary understanding of neural mechanisms of birdsong generation, recognition, its underlying neuronal pathways and discuss how predictive coding theory can be applied.

Keyword(s): predictive coding theory, error coding, Bayesian inference, sequence generation, vocal sequences, generative model

Introduction

With the advent of the 21st century, the field of neuroscience is witnessing a deluge of data. At this crucial point in time, the need for a global or unified theory(s) is imperative. To formulate a unified theory of sorts it is not enough to stick to conventional methods and concepts. One has to not only push boundaries but also cross them to solve the neural computation problem. The free-energy principle for adaptive systems, which has its roots in thermodynamics, is a framework closest to providing a unified account of how the brain works, specifically perception, action, and learning. This framework operates at the fundamental levels connecting information theory, physics, and biology. Dr. Karl Friston was the first one to connect this framework to problems in brain science. However, many people now conform to the brain as being a Bayesian Inference machine ^1–8. Such a model of the brain seems promising. Predictive coding theory an off-shoot of the free-energy principle has gained a lot of traction and for all the right reasons. The theory has been in a lot of human and animal models to study an array of interesting behavior not seldom in the songbird model system. The songbird model system is a unique model system that aids in investigating fundamental questions about neural information processing, learning, perception, sensorimotor learning, and human speech development to name a few. This review attempts to cover the basics of the predictive processing framework and try to convince the reader how differently new questions be asked and answered using the songbird model system in the light of predictive coding theory.

Basis of Predictive coding theory

The conventional view of the brain dictates the neocortex is a densely connected, feed-forward feature decoder. However, as the Bayesian inference perspective gains traction, the functionality of neocortex circuitry can be modeled using a more efficient framework, the predictive coding theory. There is a considerable amount of anatomical, physiological, and functional evidence supporting the predictive coding theory. ^5,9–11.The predictive coding theory can be thought of as an extension of the representationalist framework. A representationalist framework would be the traditional view of hierarchical information processing in the cortex that emerged from ¹². Every level in the hierarchy is responsible for encoding certain features from the stimuli which later get combined at the highest cortical areas to form the entire picture. For example, in the case of the visual cortex, let us assume the first visual processing area detects lines, this visual stimulus stream is passed on to the next area where shapes and detected, at the next level encodes fine textures, all this gets combined to perceive a face. This is a representationalist view. The key point here is to note that the flow of information processing is bottom-up only. This is in contrast (and extension) of the predictive processing view, where the flow of information processing is both bottom-up and top-down. According to the predictive coding, the higher cortical level compare their responses with the lower cortical level neurons and a prediction error is computed, which is what is used to update the previous representation and thus a clear perception is formed. See fig2. In computational terms, the probability distributions of the states in the word causing sensations or in other words belief is what gets encoded in the neural activity. This theory formulates a biologically plausible scheme for updating beliefs about states of the world using sensory samples. ^13–15

There has been increasing experimental evidence in the favour of predictive coding theory. Fig1 taken from ¹⁶reviews the anatomical and information-processing aspects of the brain that can be significantly explained under the free-energy framework. Predictive coding theory ensues’ from the free-energy principle almost, like a unifying theory of sorts that gives an explanation for the inherent cortical hierarchies, efficient encoding in the neurons, synaptic connections, physiology, and learning. This theory has been tested in humans, primates (visual cortex), and mice, but not so much in the songbird model system. The songbird model system is a unique system that can give insights into sequence generation and encoding of sequence of sequences take place at the neuronal level. There have been a few studies modeling communication using songbirds, but very few experiment-based studies. This review is aimed at bridging this gap.

Free-Energy Principle

A biological agent is known to maintain an orderly state; it resists the second law of thermodynamics ¹⁷. This characteristic distinguishes a biological system from any other system in the known universe. To resist its natural tendency to disorder, the probability that the agent will be in a small number of states is much higher than the probability that it will be in the remaining states. Mathematically speaking, these states should have a low ’entropy.’ Entropy is also the average self-information or ‘surprise’ (negative log probability of an outcome). Therefore, biological agents must minimize the long-term average of a surprise to ensure their sensory entropy remains low ¹⁸

Mathematically free energy is a function of two things, the sensory states of the agent and its recognition density that is encoded by its internal states. The agent has access to both of these parameters. The recognition density is a probabilistic representation of the cause of sensory input. ^2,3,11,14,18 The mathematical equations that explain the free-energy principle are explained in Fig2

From a neuro-centric perspective, the free-energy principle has implications for understanding action and perception. Biological agents with brains minimize their free energy in two ways; they can change the sensory input by acting on the world, or they can change their recognition density by changing their internal states

The brain as a Bayesian Inference machine

From the above figure, it is clear that the brain encodes a probabilistic representation as to the conditional distribution of the causes of the sensorium. This framework helps the brain minimize free energy or surprise by updating its representation. This is known as Bayesian inference ^14,19. Evidently, the FEP supports the Bayesian brain hypothesis or the idea that the brain is indeed a Helmholtz machine¹. This explains and accounts for the perception of biological agents. But perception is just half the story. The free-energy principle and the Bayesian Inference framework that ensued dictated that the brain is constantly trying to minimize surprise and model the environment around it. Although the brain form perception according to the Bayesian inference hypothesis to avoid surprises in the environment, it does not change the sensations or their surprise. All biological agents are open systems with respect to their environment. Therefore, the environment can act on the agent to produce sensory impressions, and the agent then acts on the environment by changing its states. However, there can be another way to minimize surprise, but taking an action to re-sample the sensorium in order to match it to the predictions or representations encoded in the higher cortical areas. This is known as Active Inference²⁰. Action reduces free energy by updating the sensory inputs signals while, whereas perception reduces free energy by changing predictions^3,20.

Vocal Learning and songbird model system

Imitation is central to learning across all species. This can be observed during the vocal development of a child. Language learning models have been investigated extensively from a neuroscientific as well as computer science/machine learning perspective. The ability of vocal learning of humans is shared only by a few other vertebrates. However, oscine songbirds, i.e., Passeriformes(e.g.zebra finches, canaries, parrots, hummingbirds, and white-crowned sparrows) have a unique form of vocal communication that has remarkable parallels to human speech learning and development. Hence, songbirds serve as a good model to study sequence generation and its correlation with neural activity. It has widely been used to investigate the mechanistic and functional basis of vocal learning and any related disorders. The song learning phase is important in the life of the juvenile bird, which comprises two distinct and significant phases 1)Sensory learning and 2)Sensorimotor learning which completely depends on the auditory experience (feedback from the tutor bird). The first phase involves the bird memorizing one or more songs by listening to the tutor’s songs, who is generally the father. This phase has four important features. First, this phase lasts for a very sensitive period that lasts for about 60-90 days post-hatch. Any songs heard after this period are very poorly copied ^21,22

Due to the similarities in songbird vocal learning and human speech development, one can draw parallels between the two brain systems

Secondly, the sensory learning phase is significantly fast. The juvenile typically listens to the tutor’s song only a couple of hundred times, which accounts for a minuscule portion of their auditory experience. The third feature dictates, that this phase results in a long-lasting and permanent memory ²⁴ Subsequently, the juvenile bird is known to have an innate predisposition to certain songs and not to other songs. It has been shown that during the sensory learning phase, juvenile swamp sparrow when made to hear songs of its own species displays a strong tendency to imitate and learn the conspecific songs and retain them for its entire lifetime ²¹ This could also mean that choosing the right tutor is controlled by an innately unique auditory representation which could be encoding features for conspecific songs. In the sensorimotor phase that ensues’ the juvenile bird makes use of the auditory feedback to match its song to its memorized template. This phase ensures the “crystallization”, a process in which the song acquires a stereotyped structure, and the generation of song bouts becomes much less dependent on any more auditory feedback. During this phase, the juvenile has typically left the company of the tutor, indicating that the generation of vocal sequences takes place entirely from memory. However, even though the juvenile does not need a tutor, auditory feedback still plays a critical role during the sensorimotor learning phase. Mark Konishi ²⁴ showed that juvenile white-crowned sparrows deafened after a tutoring experience but before the sensorimotor learning phase generated highly noisy and unstructured vocal sequences. It was thus concluded that, regardless of whether the bird was exposed to a tutor learning experience or not, the juvenile tries to match its song it an internal model of the song, during the sensorimotor learning phase. This internal model thus forms an acoustic memory of the tutor song, in isolates, the model must be innately generated. This internal model, whether learned or innate is often referred to as the song “template”. In contrast to the sensory phase, the sensorimotor learning phase comprises extensive vocal experimentations and occurs relatively slowly.

Neural circuit dynamics of vocal learning

Processing of the song sequences in the songbird brain takes place in a specialized circuitry responsible for vocal learning and production. These pathways have also been shown to have analogs in the mammalian brain.

The primary auditory processing area, which is the auditory cortex, is located on the lateral side of the temporal lobe. The production, acquisition, generation, and learning of the birdsong involve two pathways as follows(refer to figure 5):

• 1.) Anterior forebrain pathway (AFP):- involved in vocal learning and composed area X (homolog to mammalian basal ganglia), lateral part of the magnocellular nucleus of the anterior nidopallium (LMAN), and the dorsolateral division of the medial thalamus (DLM)
• 2.) Posterior descending pathway (PDP):- involved in vocal production, composed of HVC and robust nucleus of the arcopallium (RA)

Lesions of HVC and the RA drastically affect the structure and organization of the song sequence, HVC is known to have catastrophic effects ²⁶ . In contrast to RA, if HVC is stimulated during a song, the song is interrupted and completely resets the song program²⁷

Electrophysiological recordings of the singing bird have confirmed a hierarchical feed-forward architecture connecting HVC to RA. However, the exact encoding of the song syllables in HVC remains open to investigation. This issue has been further tested by recording from specific RA projecting HVC neurons. Each neuron has a stereotypical firing pattern spanning 6ms, which is very sparse and occurs at a narrowly defined timepoint ²⁸

The AFP plays a role only in song acquisition, it is not involved in the production of the learned song ²⁹ Bilateral lesions of the LMAN, a part of the AFP, cause the variable song typical of young birds to become stereotyped, which indicates that LMAN plays a critical role in building the circuits’ plasticity necessary for song learning ³⁰But the underlying mechanism for this effect is not known. Ölveczky and colleagues have shown that RA projecting LMAN neurons fire in a quasi-random pattern when the young birds sing a variable song. Thus one can conclude that while the HVC–>RA neurons encode for the learned song, the LMAN–>RA neurons encode the stochasticity that brings about the variability in the output of the soundbox of the bird, which is important for imitation.

Hahnolser et al.,2002 ²⁸ pointed out that most of the RA activity involved in songs could be driven in a domino-like feed-forward activity pattern by HVC. As a population, RA projecting HVC neurons fire sparsely, and most RA neurons generate a unique pattern of bursts with respect to the projection neurons. HVC-RA neurons produce a unique sparse burst at a single time-point in the song and as a population they are active throughout the RA sequences, indicating that HVC-RA neurons encode a neuronal state within a sequence rather than coding for some syllable or motif. Long et al 2008 ³¹ further confirmed that as a population HVC neurons form a domino-like chain of activity that control the song sequences. A subsequent paper by the Long group ³² carefully tested and compared evidence for the synaptic chain model and ramp-to-threshold model. There was absolutely no evidence for the latter. It is also essential to note that the temporal structure of the song is controlled by calcium-mediated bursts through a synaptically connected chain model. The HVC nucleus has both excitatory and inhibitory populations of neurons. The exact balance of these neural dynamics was studied by Kosche ³³

They confirmed that there is a fine-tuned structured excitatory and inhibitory balance that gives us the stereotypical pattern in neural activity. Fig8 depicts the song bout and firing patterns of HVC neurons of a zebra finch.

After knowing the basic neurobiology of the songbird, it becomes imperative to understand what are the tasks involved in vocal learning and vocalizations. The first one is to generate song motor commands and the second one is to vary these motor commands to match the song template. Matching the vocalizations with the template song from memory requires auditory feedback. To model this process of evaluation of learned song to memorized song, one can make use of a comparator circuit. See Fig7. When the two different inputs enter, the comparator emits a +1 or 0 signal that causes the song to get updated, as a result minimizing the error between the song and the memorized model. Hence, an internal model of how the songbird’s brain represents sequences of songs to generate orderly motor movements is a question that would give us insights into understanding if there is a fundamental information processing framework that biological agents with brains implement.

7. Discussion

Understanding the generative models in the songbird model system is worth pursuing. The critical period of sensorimotor learning brings with it bout-to-bout variability. Invesigating if and how this variability constitutes or aids plasticity and learning at the functional and synaptic levels would be an important stepping stone. Auditory feedback has a vital role to play in vocal learning. Previous studies have shown a special population of neurons are responsible for decoding information from the auditory feedback and make use of it in learning and plasticity. An important next step will be to determine whether and how feedback-related changes in the activity of these neurons are harnessed for modifying vocalization. The nature of the comparator which models how a variable song is crystallized into the final template remains poorly understood at the neuron level. One hint that LMAN may function as the output of the comparator is that it is necessary for both acute song variability and song plasticity. An important direction to be pursued can be to find out if LMAN is functionally active during sensorimotor learning to correct errors from the auditory feedback. In the light of predictive coding, one needs to investigate a unique cohort of prediction error neurons in the birdong motor pathway. There may not be a single area responsible for to function as a comparator between the stimilus and a previosly memorized song but several levels in the pathway may be involved. These and more such questions are worth investigating as answering them will help fill the gap in our knowledge repertoire. A good understanding of neurophysiology will in turn help us build better models of neural information processing.

References

1. Dayan, P., Hinton, G. E., Neal, R. M. & Zemel, R. S. The Helmholtz Machine. Neural Comput. 7, 889–904 (1995).
2. Friston, K. A theory of cortical responses. Philos. Trans. R. Soc. B Biol. Sci. 360, 815–836 (2005).
3. Friston, K., Kilner, J. & Harrison, L. A free energy principle for the brain. J. Physiol.-Paris 100, 70–87 (2006).
4. MacKay, D. J. C. Information Theory, Inference, and Learning Algorithms. 640.
5. Mumford, D. On the computational architecture of the neocortex. Biol. Cybern. (2004) doi:10.1007/BF00202389.
6. Neisser, U. Cognitive psychology. (Appleton-Century-Crofts, 1967).
7. Rao, R. P. N. & Ballard, D. H. Predictive coding in the visual cortex: a functional interpretation of some extra-classical receptive-field effects. Nat. Neurosci. 2, 79–87 (1999).
8. Vilares, I. & Kording, K. P. Bayesian models: the structure of the world, uncertainty, behavior, and the brain. Ann. N. Y. Acad. Sci. (2011) doi:10.1111/j.1749-6632.2011.05965.x.
9. Adams, R. A., Shipp, S. & Friston, K. J. Predictions not commands: active inference in the motor system. Brain Struct. Funct. (2012) doi:10.1007/s00429-012-0475-5.
10. Bastos, A. et al. Canonical Microcircuits for Predictive Coding. Neuron (2012) doi:10.1016/j.neuron.2012.10.038.
11. Friston, K. J. Hierarchical Models in the Brain. PLoS Comput Biol (2008) doi:10.1371/journal.pcbi.1000211.
12. Hubel, D. H. & Wiesel, T. N. Receptive fields of single neurones in the cat’s striate cortex. J. Physiol. 148, 574–591 (1959).
13. Friston, K., Thornton, C. & Clark, A. Free-Energy Minimization and the Dark-Room Problem. Front. Psychol. 3, (2012).
14. Friston, K. J. The free-energy principle: a unified brain theory? Nat. Rev. Neurosci. (2010) doi:10.1038/nrn2787.
15. Friston, K. J. Is the free-energy principle neurocentric? Nat. Rev. Neurosci. (2010) doi:10.1038/nrn2787-c2.
16. Friston, K. The free-energy principle: a rough guide to the brain? Trends Cogn. Sci. 13, 293–301 (2009).
17. Feynman, R. (1972) Statistical Mechanics A Set of Lectures. W.A. Benjamin Inc., New York. - References - Scientific Research Publishing. https://www.scirp.org/(S(i43dyn45teexjx455qlt3d2q))/reference/ReferencesPapers.aspx?Ref erenceID=1868135.
18. Friston, K. J., Trujillo-Barreto, N. & Daunizeau, J. DEM: A variational treatment of dynamic systems. NeuroImage 41, 849–885 (2008).
19. Knill, D. C. & Pouget, A. The Bayesian brain: the role of uncertainty in neural coding and computation. Trends Neurosci. 27, 712–719 (2004).
20. Friston, K. J., Mattout, J. & Kilner, J. Action understanding and active inference. Biol. Cybern. (2011) doi:10.1007/s00422-011-0424-z.
21. Marler, P. Song Learning: The Interface between Behaviour and Neuroethology. Philos. Trans. Biol. Sci. 329, 109–114 (1990).
22. Immelmann, K. Song development in the zebra finch and other estrildid finches. undefined (1969).
23. Doupe, A. & Kuhl, P. Birdsong and human speech: common themes and mechanisms. Annu. Rev. Neurosci. (1999) doi:10.1146/ANNUREV.NEURO.22.1.567.
24. Sparrows learn adult song and more from memory - PubMed. https://pubmed.ncbi.nlm.nih.gov/17834587/.
25. Nottebohm, F. The Neural Basis of Birdsong. PLoS Biol. (2005) doi:10.1371/journal.pbio.0030164.
26. Nottebohm, F., Stokes, T. M. & Leonard, C. M. Central control of song in the canary, Serinus canarius. J. Comp. Neurol. 165, 457–486 (1976).
27. Vu, E. T., Mazurek, M. E. & Kuo, Y. C. Identification of a forebrain motor programming network for the learned song of zebra finches. J. Neurosci. Off. J. Soc. Neurosci. 14, 6924–6934 (1994).
28. Hahnloser, R. H. R. An ultra-sparse code underlies the generation of neural sequences in a songbird. Nature (2002) doi:10.1038/nature00974.
29. Forebrain Lesions Disrupt Development But Not Maintenance of Song in Passerine Birds. https://www.science.org/doi/10.1126/science.6719123.
30. Scharff, C. & Nottebohm, F. A comparative study of the behavioral deficits following lesions of various parts of the zebra finch song system: implications for vocal learning. J. Neurosci. Off. J. Soc. Neurosci. 11, 2896–2913 (1991).
31. Long, M. A. Using temperature to analyse temporal dynamics in the songbird motor pathway. Nature (2008) doi:10.1038/nature07448.
32. Long, M. A. Support for a synaptic chain model of neuronal sequence generation. Nature (2010) doi:10.1038/nature09514.
33. Kosche, G., Vallentin, D. & Long, M. A. Interplay of Inhibition and Excitation Shapes a Premotor Neural Sequence. J. Neurosci. 35, 1217–1227 (2015).