Generating Coherent Patterns of Activity from Chaotic Neural Networks

Sussillo, D., Abbott, L.F. (2009). Generating Coherent Patterns of Activity from Chaotic Neural Networks. Neuron 63 544-557.

I wanted to get into the Larry Abbott stuff as it seemed to pertain to the local bending model I was envisioning. So this paper covers these ideas of using chaotic random networks as the basis for learning arbitrary patterns. Essentially because of the high dimensional space the pattern is easy to read out with some simple linear combination. They expand upon previous work and make the model more complex and incorporate "FORCE" learning.

The idea of FORCE learning is that you start with a fast learn (pretty easy because its just a linear thing), but you don't immediately try to get the error all the way to zero. You keep a small amount of error that then decreases slowly. The reason for this is that the networks tend to destabilize and ruin the output. Part of the key of keeping the network in line is that the output feedsback into the network, and the chaos gets overcome by the pattern. Essentially with no stimuli the network tends to be chaotic, but when given a periodic stimuli to learn, the network loses its chaos.

From the examples its quite good at learning arbitrary patterns. They do an interesting PCA analysis of the network after its learned a particular pattern. The network is initiated with the same random connectivity, and then taught to learn the same pattern. The readout weights are initiated at different random strengths. Once the pattern is learned, the major components (the first few PCs) converge to the same point, regardless of where the readout weights were initialized. The later components end up being at a different place.

The learning performs best right on the verge of chaos. By tuning a network parameter (essentially the average synaptic strength), they can vary the amount of chaos in the network. It performs poorly at learning tasks when the network is not in the chaotic regime, but it also cannot work when the network is too chaotic. The learned pattern has to be able to overcome the intrinsic network chaos, and the network needs some chaos so that it can learn the pattern.

Finally they show a network that can do interesting input-output transformations. They add some control inputs, which essentially connect to the network randomly and train the outputs when the appropriate control inputs come on.

(A) Network with control inputs used to produce multiple output patterns (synapses and readout weights that are modifiable in red).
(B) Five outputs (one cycle of each periodic function made from three sinusoids is shown) generated by a single network and selected by static control inputs.
(C) A network with four outputs and eight inputs used to produce a 4-bit memory (modifiable synapses and readout weights in red).
(D) Red traces are the four outputs, with green traces showing their target values. Purple traces show the eight inputs, divided into ON and OFF pairs associated with the output trace above
them. The upper input in each pair turns the corresponding output on (sets it to +1). The lower input of each pair turns the output off (sets it to  1). After learning, the network has implemented a 4-bit memory, with each output responding only to its two inputs while ignoring the other inputs.


The 4-bit memory is a pretty cool example. For each bit there is an on and off input pulse, which flips the state of the corresponding readout unit. So in the network there are 16 fixed points corresponding to all possible bit combinations.

Then they have a much more complex example of the network implementing running and walking like behaviors. They got some motion capture data and trained the network to mimic the joint angles for 95 different joints. And the network was able to learn both walking and running.

The network synaptic structure is linear. Firing rate model that goes through a tanh output non-linearity

Leech mechanosensory model - Fractional derivative

Ok, now I've incorporated the T cells, but I thought a simple linear filter would enable rates like in the animal. But it wasn't quite right -- the T cells didnt fire on the off phase of the steps. So instead of doing the linear filters, I implemented fractional derivatives for the "filters". So now the P cell filter is the 0.3 order fractional derivative of the sensory signal, and the T cell filter is the 1.6 order fractional derivative.

Figure 1: Sensory signal, P and T cell currents based on fractional derivative of signal.

Figure 2: receptive fields of P and T cells.

Figure 3: P cell responses to sensory signal.

Figure 4: T cell responses to stimulus

Leech mechanosensory model - sensory representation

Ok, I'm going to spend a few days just working on this new model of the leech sensory system.

It starts with the touch as being a set of "pixels" that go around the skin and the value for each pixel is the instant pressure on the skin. So here is like what the touch stimuli look like:

Figure 1: The upper panel is the actual touch stimulus. Here there are just 10 pixels that go around the circumference of the body. We are going to transform this touch stimulus to a train of P cell spikes by both incorporating the receptive field and a temporal filter. In the second panel is the touch stimulus convolved with the P cell temporal filter. This is just a very stupid filter right now that is supposed to fit the P cell a little, but is nowhere near accurate. The final panel is the actual currents that will be injected into each P cell (note that there are 4 rows for each P cell).

Figure 2: Here the P cells are just single compartment IF neurons, and this is there responses to the touch stimulus shown in Figure 1.. The first panel is the voltages, with threshold of -58 mV and Reset of -62. The second panel is the actual spike times of the 4 P cells, and the final panel is the ISI for all 4 P cells. 

Figure 3: This is the P cell temporal filter. 

Multi-subject dictionary learning to segment an atlas of brain spontaneous activity

Varoquaux, G., Gramfort, A., Pedregosa, F., Michel, V., Thirion, B. (2012). Multi-subject dictionary learning to segment an atlas of brain spontaneous activity. Information Processing in Medical Imaging 6801.

Ok so a lot of the registration algorithms for fMRI might be relevant for my VSD data. This paper was recommended to me by Stephan Gerhard (from NS&B).

They take an approach that combines ICA with dictionary learning.

Start with a linear signal decomposition. Y is data time series (n timeponts x p voxels). V describes functional processes or structured measurement artifacts with k spatial latent variables (p x k). U describes the time series of the latent variables (n x k). Y = U * V

So they are setting up the problem to create a hierarchical generative model of the data, and they have an optimization procedure to find U and V.

The algorithms are written into the text as pseudo-code, varoquaux's website: http://gael-varoquaux.info/, and maybe some related code: https://github.com/GaelVaroquaux/canica

Theres also a package NiPy that has a lot of neuro-imaging algorithms. There may be some relevant algorithms there for my data. And also: http://scikit-learn.org/stable/

The rest of the paper is validation, but there are some other good references. I need to dive into the code to start getting some real insights into the applicability of this to my data.

Arbitrary heteroclinic orbits

Yeah so after hearing about crawling a bunch of times up here in MBL it seems clear that heteroclinic like orbits are more closely related to how CPGs work. This is in contrast to the Shilnikov style of doing CPGs, as they rely on intrinsic dynamics and pacemaker neurons -- effectively making CPGs out of coupled oscillators. The heteroclinic orbits don't require coupled oscillators and have the nice properties of being able to hold up the period in certain places like real CPGs. Plus, if you looked at a single CPG neuron that was part of a heteroclinic orbit, then you wouldn't see any intrinsic oscillatory dynamics. This seems to be the case most of the time for neurons.

So the question is how to make arbitrary types of heteroclinic orbits with neural-like models. There would have to be some population that begins to turn on and starts off as an attractor through some kind of recurrent feedback. But once the population turned on enough it acts like a source and the dynamics flow towards the next population. Each population would be a fixed point, but as you approach from one direction its attractive, and then it sends you off in another direction. But how to make this type of system is the question.

Dan Gerschwin

He came and gave a talk today at MBL. It was about genetics -- well really the "transcriptome", as reading all of the RNA is apparently much easier than the genome or the proteome. It was very interesting and the analysis he was doing was quite remarkable.

Most people look at up or down regulation of RNAs and try and correlate that with phenotype. He was actually looking at how different RNAs were correlated together, and then relating that to phenotype. Basically based on the correlation of different RNA expression levels he could build a distance matrix and then do graph-based analysis. He was showing how different RNAs cluster into "modules" and that modules were organized in reasonable ways -- many of the modules were related to the cell types and other pretty obvious differences. He had some cool ways of clustering and dimensional analysis that could be relevant to thesis work.

Neural pointers

So how does info get routed into working or hippocampal memory? Pointers! Working and hippocampal memory are not encoding the information about a memory, but rather pointing to PTV where the information is stored. These pointers are stored in the sequence of spikes during each theta cycle. When Frontal wants to remember something, then it burst activates the neurons which leads to some plasticity. This plasticity is bi-directional. The PTV neurons that are currently active reinforce together (so that the memory can be recalled more easily ala a hopfield net/local minima), and then reinforce with a hippocampal sequence. The hippocampal sequence also reinforces with all of PTV, so that when the hippocampal sequence gets activated all of the info in PTV will be reactivated as well.

Lets think about working memory a little more. I vaguely remember some talks/papers I've read about how working memory would be implemented by the brain. there is a region in frontal cortex where a slow oscillation similar to theta dominates. During this slow oscillation, about 7 neural pointers can be stored (in the human brain). Each pointer was reinforced with some frontal burst feedback, so that each of the pointers can reactivate the PTV that has all of the information. You can imagine how the working memory structure can just maintain the 7 pointers in its slow oscillation through some sort of recurrent activation and plasticity or through that STP mechanism. This is how the working memory is maintained over longer time-scales. Then the different pointers being stored in working memory can be reactivated by another frontal area (another burst signal or something), which brings back the information stored in PTV.

Hippocampus is basically the same, but the pointers are built over a slow timescale (during each theta cycle), and then a memory over a larger time-scale can be recalled more quickly by activating a sharp-wave ripple.

PTV and Frontal

So we are starting to build this picture about what the PTV (Parietal-Temproal-Visual) cortex is doing and fitting it in with the rest of the brain. What it seems is that PTV is encoding/representing/building the current and recent state of the world. When you open your eyes and see the world around you, recognize the objects, hear the sounds, the 3D structure, etc. all of this information is being represented in PTV. . There is a tremendous amount of information continually being encoded by this circuit, and this information is the immediate state of the mind you are experiencing.

Now, this is too much information to make sense of all at once. And this is where frontal cortex comes into play. Frontal cortex essentially feedsback to all of PTV (via layer 1). Frontal cortex enhances the activity in subsets of PTV and can route subsets of PTV into working or long-term memory. To encode a long term memory, only the parts of PTV that are strongly modulated by top-down attentional feedback make it. Essentially attentional feedback activates the neurons and gets them to burst. This doesnt really change the information anywhere in PTV, but enhances is it. The burst is a signal that carries the same info about the sensory environment but tells other structures to operate differently. So to work something into say hippocampal memory (I guess this is more like medium-term), frontal cortex burst synchronizes every region of the brain it wants to remember. The entire hierarchy can be encoded in the local cortical circuits, and the hippocampus plays the role as the pointer and relates the cortical information being saved with a time-place pointer in the hippocampus. When the memory is recalled, the hippocampus can be reactivated which then reactivates the cortical areas.

PTV is creating a hierarchical model of the world from all of the information from the senses. This model is actually non-temporal (ugh, confusing in this case im referring to temporal as in time) -- it has temporal structure, but it is time-invariant. Your phone stays your phone, etc. Frontal cortex is more in charge of time. Frontal cortex is building a hierarchy to predict the future at different temporal scales. Frontal cortex is pulling information about the current state of the world from temporal cortex and routing it through working and long-term memory. Frontal routes PTV into working memory, and can reactivate hippocampal memory to route into working memory to make better predictions about the future.

mixed selectivity

nature paper about "mixed selectivity" neurons in PFC. Sounds a lot like population coding...

http://www.nature.com/nature/journal/vaop/ncurrent/full/nature12160.html

Tirin Moore

Just went to a talk by Tirin Moore from stanford. He studies visual attention and was doing physiology work in FEF and V4. He had a lot of interesting data, but it was kind of all over the place.

His model had neurons with D1 receptors in 2/3 of FEF, and these were the ones that fedback to V4. He showed through antidromic AP annihilation that it was mainly the persistent firing cells that fedback to V4. He showed that modulation of D1 and D2 receptors in FEF have the same effect of behavior -- biasing the saccades to the RFs in FEF (although it was D1 antagonist and D2 agonist, which he said didnt matter because they have "U" shape properties). D1 changes had effects, however, on the properties of V4 neurons -- typically looking like enhanced attention. D2 had no real effect on V4, but had the behavioral result. D2's seemed to be more involved in the motor part of the saccade, and D1 in the attentional part.

It seemed like a clear case of Modulatory type of feedback projections. The neuron's were enhanced by FEF feedback, but required FF stimulation. The fact that the feedback was from the persistent firing neurons made sense to me -- keeping the modulation up in preparation for the stimulus to appear. Although, he said that the selectivity was narrowed, which goes against the kind of gain control modulation that I think of. However, this may just be because you're altering FEF in a non-sensical fashion, and that causes selectivity changes. And he was measuring V4 selectivity with oriented bars.

PCA ICA course

I'm also finally done teaching the PCA/ICA course in the student comp class. I really like how serious the class is, I guess classes are a lot better when you have all the best kids and their motivated -- and also you aren't really responsible. I really enjoyed it, because it forced me to learn ICA pretty hard. And now I feel a lot more confident about it, and i will seriously be able to do some stuff with my data with ICA.

I like a lot of the ideas in the homework, but it definitely can be better. Honestly it is pretty easy right now, but some parts are confusing. But I really enjoyed people's answers and i thought that it was tricky in the right ways. It made me happy to see their answers. 

The ICA part was hard because I was a jerk and did it in python, and the mac kids had problems. But, I think its a really good exercise and theres a lot more to explore. I had them just add noise and see what happened, and i made algorithms to match up the ICs and print out the cost. i wanted them to come up with some more things, but nothing too fancy. They could also change the mixing matrix such that they had more or less microphones to actually see what happens.

I realize now that I had no idea how to use fastica for the image analysis. The problem was that it is not like PCA at all, and I just assumed that it was. It is interesting because the ICs are scalar-invariant. So they come out in no particular order and there's nothing that says how important they are -- there are no eigenvalues. I don't quite understand why they don't get sorted by whatever it is they're optimizing -- their non-gaussianity. Fastica is supposedly doing some optimization, so why don't each of the components have an optimization value, can't you sort by that?

The other useful thing would be to find the mixing matrix. It wouldn't really be that hard -- its basically fitting the microphones with the ICs, but maybe there are better and worse ways... I just did a linear regression to see which ICs matched up with the original components. I guess if you just matched them to the data, wouldn't it come out to an exact mixing matrix? 

Yi Zuo

Hosted Yi Zuo this week. She was the seminar speaker and talked about how dendritic spines form during learning tasks. I took away a few extra things from her talk -- one was that the spines stopped forming as much once the behavior had plateaued, kinda like perceptron learning rule (once you get it right stop changing). The other was that spine growth was present shortly after training, but there was no spine elimination. But, this was only true if the imaging was done immediately after -- imaging a day after then there was increased spine elimination to compensate for the growth. It just has to be something to do with sleep.

I actually had a really good time talking with her and learning about her life. I was very worried because she was quite Chinese and I was afraid it was going to be hard to communicate. But she was very talkative, maybe even too talkative, but I was happy because not too many awkward silences. I couldn't believe how energetic she was in general. I was exhausted and I was doing nothing for most of the day, she had talked to so many people: Mayford, Jim Connor, Lunch (Jeremy Biane, Tom Gillespie, ...), Zheng, Scanziani, Komiyama, the talk, dinner (andy, sam, adam, jeremy, heidi, kathleen). And then jeremy gave her a laptop talk about his data. Damn she was a trooper.

Functional mapping of single spines in cortical neurons in vivo

Chen, X., Leischner, U., Rochefort, N.L., Nelken, I., Konnerth, A. (2011). Functional mapping of single spines in cortical neurons in vivo. Nature 475.

Two-photon (LOTUS) imaging of individual spines in auditory cortex. Present tone stimuli at different frequencies and volumes. Spines show a variety of tuning properties, but no clear clustering on dendrites is observed.

Figure 3 | Frequency tuning and heterogeneous distribution of individual
active spines. a,Upper panel: two-photonimage of a dendritic segment of a layer
2/3 neuron (average of 6,250 frames). Lower panel: calciumresponses (average of
five trials) from two spines (S) marked by red arrowheads in the upper panel,
during 11 pure tones (from 2kHz to 40 kHz at 0 dB attenuation). Two
neighbouring spines indicated by blue arrowheads did not respond to any of the
11pure tones.b,Upper panel: frequency tuning curve of thenarrowly tuned spine
S1 shown in a. Data points are the mean values of response amplitudes from
five trials. Lower panel: average tuning curve normalized to the highestamplitude
(n538 spines, 10 neurons). c,Upper panel: frequency tuning curve of thewidelytuned
spine S2 shown in a. Lower panel: average tuning curve, normalized to the
highest amplitude (n546 spines, 10 neurons). Error bars in b and c show s.e.m.
d, Distribution of frequency tuning widths (DFrequency) of pure-tone-activated
spines (n584 spines, 10 neurons). e, Heterogeneous distribution of pure-toneactivated
spines along dendrites. Cartoons of dendritic segments from four
neurons, with numbers indicating the effective frequencies for each active spine.
Narrowly tuned and widely tuned spines are indicated by red and blue dots,
respectively. The neurons correspond to, respectively, neuron 25, neuron 27,
neuron 29 and neuron 30 in Supplementary Table 1. f, Plot of the most effective
frequency of a given spine versus themost effective frequency of its nearest active
spine (seeMethods).Dots along the red line correspond to pairs of spines that had
the same most-effective frequency (n569 pairs, 24 dendrites, 10neurons).
g, Plot of the distance between neighbouring active spines versus the difference
between their respective most-effective frequencies. For each pair of spines, the
reference spine (blue circle) was defined as the left spine and the test spine (red
circle) was defined as the neighbouring active spine on the right. The
measurements were performed sequentially from left to right in each dendrite
(n551 pairs, 24 dendrites, 10 neurons). Dots along the red line correspond to
spine pairs that had the same most-effective frequency. Numbers on the right
indicate distance ranges between pairs of spines with a difference between their
most effective frequencies of 0–1, 1–2, 2–3 and 3–4 octaves.

ICA

The motivation behind developing ICA can be illustrated by a classic problem -- the "Cocktail Party" problem. The basic idea is that you are at a big party and you hear the voices of many different people talking. Your ears are picking up the sum of all of these people's voices and your brain has the task of separating out each of the voices. Each ear receives a slightly different signal -- say your left ear is closer to speaker 1, and the other ear is closer to speaker 2. The left ear would receive a stronger signal from speaker 1 and a weaker signal from speaker 2. Mathematically each ear is just receiving a weighted sum of all of the signals.

x = M*s

So x is what your ears pick up, s are the speakers at the party, and M describes how the signals from each of the speakers get mixed together to make the observed signal. This is the "Mixing Matrix".

The assumptions about the problem that we are trying to solve is that the signals that we observe are actually mixtures of signals that are independent -- the speakers talking at the party are not influencing each other. So how do we pull out the independent signals from the observed data in x? What exactly do we mean by independence?

There are some strict probabilistic definitions of independence -- namely p(a,b) = p(a)*p(b). These p's describe the full probability distributions of the signals, and independence means that the chance of observing a and b together is equal to the product of observing each of them independently. If this holds true across the entire probability distribution, then the signals are independent. However, this definition is almost useless for any realistic application, because it would require accurately estimating the complete distributions and the complete joint distribution of the inputs. 

Correlation is something that you could easily calculate practically, but no correlation does not necessarily imply independence. Independence, however, does imply that no correlation. A sine wave and a cosine wave plotted against each other would create a circle -- data in the shape of a circle would clearly be uncorrelated. However, these data are obviously not independent.

A big intuition about how to determine independence comes from the Central Limit Theorem. This is one of the most fundamental ideas in probability and statistics. The central limit theorem states that the summation of many independent random events will converge towards a gaussian. It doesn't matter what the probability distribution of the individual events are -- they can be uniform, binary etc, but adding many independent elements together will produce a Normal distribution. A nice example is rolling dice. If you roll one die, then there is a uniform probability that it will land on one of the numbers 1-6. If you roll two die, then the sum of the die no longer has a uniform probability -- the sum is most likely to be 7, and least likely to be 2 or 12. As you add more and more dice the probability distribution gets closer and closer to looking like a bell curve -- a Normal/Gaussian distribution. The Central Limit Theorem is why a Normal distribution is called Normal -- we see these distributions all over the place in the real world because independent events when summed together take this shape. 

For ICA we are assuming that several independent components are being mixed together to create our observed signals. The Central Limit Theorem states that if we mix independent components then we get something that is more Normal than the original components. Thus the goal of ICA is to find a projection of the mixed signals that is the LEAST Gaussian or the most non-gaussian. ICA thus becomes an optimization problem where we are trying to find a projection of the data that gives us the most non-gaussian components.

In order to solve the optimization problem, we must define what we mean by non-gaussian -- how do we measure non-gaussianity? There are several possible ways to estimate the gaussianity of data, and each have different trade-offs. Estimators can be optimal but require a lot of data or they can be efficient but biased. The field is not fully set on the "best" way of estimating non-gaussianity, but there are a few ways of doing it which stem from probability theory and information theory. In reality these different metrics are actually quite similar, and probability theory and information theory are actually mathematically linked -- several studies have shown how estimators derived from probability theory are actually identical to some estimators derived through information theory. There is in fact an underlying mathematical structure that links both fields.

One of the most prominent and conceptually simple ways of measuring non-gaussianity is to use the "Kurtosis". The kurtosis is related the 4th order moment of a probability distribution (the first is the mean, the second is the variance, the 3rd is known as skewness). The kurtosis is the peakiness of the distribution. There are infinitely more moments, and all are defined by raising the expected data to different powers (the nth moment is E(x^n), where E is the Expectation operator). Kurtosis is technically the "excess" of the 4th moment, and is usually compared to the Kurtosis of a Gaussian distribution. A Gaussian distribution has a 4th moment of 3-sigma^4 (where sigma is std). The kurtosis is defined relative to a Normal distribution with variance of 1, and so you just subtract out the kurtosis of a Normal distribution with the same variance as the data -- K = E(x^4) - 3. 

ICA

I'm teaching ICA in the comp-neuro student course and I'm studying up on it today. I'm just going to try and make an outline of the important points to cover. This tutorial is really good, and will serve as the basis.

Here are the main points I want to cover:

1. Motivation
1. The cocktail party problem
2. Definition -- matrices and data
3. Ambiguities
4. Illustration
2. What is independence?
1. Central limit theorm
2. Nongaussian is independent
3. Estimators of nongaussianity
1. Kurtosis
2. Negentropy
3. Others
3. Preprocessing for ICA
1. Centering
2. Whitening
3. PCA
4. Fast ICA
5. Neuroscience Applications
1. EEG
2. Imaging