Swim Oscillator Neuron Review

Collecting papers on identified neurons from previous work.

Briggman, K.L., Kristan Jr., W.B. (2006) Imaging Dedicated and Multifunctional Neural Circuits Generating Distinct Behaviors. J. Neurosci 26(42):10925-10933. 

This is a nice one of CV. You can see it is slightly lagging the DP oscillation, but I'm not entirely sure that the ganglion is 12. My guess is its 10 and he's recording DP 12, so the DP spikes are a little further behind.

 And then he gets some nice simultaneous 208 and 255 recordings. Again this matches really nicely with my data as you can see 255 slightly leading 208. This is reflected in that 255 is in the red phase and 208 is in the purple phase in my data.

And he also recorded from 3 and 4, but these are on the dorsal face.

Weeks, J.C., Kristan Jr., W.B. (1978) Initiation, maintenance and modulation of swimmming in the medicinal leech by the activity of a single neurone. J. exp. Biol. 77: 71-88.

Here's the position of 204, which matches nicely

Depolarizing cell 204 leads to the initiation of the swim oscillation.

And then here is 204 during swimming

Ok, so I put 204 in the yellow phase, which I think this matches nicely. It is hard to tell because this is DP 12, but you can see how much the phases shift by comparing 204 (10) with 204 (11). The way I have it, 204 is in the yellow phase, which means its depolarization should slightly lead the DP nerve. It would be unclear, because the duty cycle of 204 is pretty long, which can maybe slightly alter the phase, but I'm fairly happy with it. 

Weeks, J.C. (1982) Segmental specialization of a leech swim-initiating interneuron, cell 205. J. Neurosci. 2(7): 972-985.

Another paper from weeks looking at cell 205, which I don't see how to possibly differentiate from cell 204.

Although, wow it looks like S has a swim oscillation. She is showing that S is connected to cell 205, as well as cell 205 getting a bunch of input from sensory neurons -- sounds like its part of preparatory network.

But here is a simultaneous 204, 205 recording:

So, these are very similar. Maybe 205 is 263/264?

205 is the only one missing from my data set, but this is only in segment 9 according to Weeks, 1982. 

"Unlike other swim neurons which are segmentally repeated, cell 205 generally is present only in segment 9, and numerous lines of evidence suggest that it is, in fact, a segmentally differentiated homolog of cell 204."

Friesen, W.O., Hocker, C.G. (2001) Functional Analyses of the Leech Swim Oscillator. J. Neurophysiol. 86:824-835.

This schematic is in a lot of Friesen's papers. I think most of the neurons involved in swimming that Friesen studies, however, are on the dorsal face.

And here's an intracellular of DI-1 during swimming:

Cang, J., Friesen, W.O. (2002) Model for Intersegmental Coordination of Leech Swimming: Central and Sensory Mechanisms. J. Neurophysiol. 87:2760-2769.

Another one of these phase diagrams from Friesen:

The rest of this paper is about modeling the swim behavior.

Brodfuehrer, P.D., Thorogood, M.S.E. (2001) Identified neurons and leech swimming behavior. Progress in Neurobiology 63: 371-381.

Here is a diagram of the cascade that causes swimming and a recording of DE3

More circuits and recordings. The legend says panel A is from Friesen 1989, and panel B is from Nusbaum 1987, which I need to find.

Not sure what SIN1 is, this is from Brodfuerher and Burns 1995:

Taylor, A.L., Cottrel, G.W., Kleinfeld, D., Kristan Jr., W.B. (2003) Imaging Reveals Synaptic Targets of a Swim-Terminating Neuron in the Leech CNS. J. Neurosci. 23(36):11402-11410.

Here they use the VSDs to find cells that are targeted by Tr2, which is a head cell that terminates swimming. He identifies two cells - 256 and 252 that are targeted by Tr2. These cells seem to also terminate swimming, but not really clear if they are oscillators.

Here are ephys recordings, but it doesnt seem that 256/252 are oscillators

Kristan Jr., W.B., Calabrese, R.L., Frieson, W.O. (2005) Neuronal control of leech behavior. Progress in Neurobiology 76: 279-327.

Pretty detailed review. Good source to cite as leech cells being homologous: "Most midbody ganglia have the same neurons and locations of the soma."

7. The nervous system is iterated, with homologous neurons found in most, if not all, 21 segmental ganglia (Fig. 1). So despite having more than 10,000 neurons, the functional unit (i.e. the number of different kinds of neurons) of the leech CNS is relatively small. For instance, there are only 400 neurons per segmental ganglion (Macagno, 1980), and most of these are paired. Thus, in essence, the segmental nerve cord (roughly corresponding to the spinal cord in vertebrates) consists of 42 copies (one on each side of 21 segments) of a basic unit of 200 neurons.

And another version of the circuitry figure, with colors!

Brodfuehrer, P.D., McCormick, K., Tapyrik, L., Albano, A.M., Graybeal, C. (2008) Activation of two forms of locomation by a previously identified trigger interneuron for swimming in the medicinal leech. Invert. Neurosci. 8:31-39.

Mostly about Tr1. But here's a CV:

And it looks like Retzius has a clear swim oscillation. Hard to tell its phase from this picture, however.

Friesen, W.O., Kristan, W.B. (2007) Leech locomotion: swimming, crawling, and decisions. Current Opinion in Neurobiology 17:704:711.

This paper just wanted for this figure to cite the "Do something!" network.

Friesen, W.O. (1989) Neuronal control of leech swimming movements I. Inhibitory interactions between motor neurons. J. Comp. Physiol. A. 166:195-203.

Ok this is Otto's original papers on swimming. The first one is about the motor neurons and their connections:

Here's the classic about the motor neuron coupling:

Friesen, W.O. (1989) Neuronal control of leech swimming movements II. Motor neuron feedback to oscillator cells 115 and 28. J. Comp. Physiol. A 166:205-215.

Yeah so most of the diagrams that I'm interested in originally come from this paper.

Nusbaum, M.P., Friesen, W.O., Kristan Jr., W.B., Pearce, R.A. (1987) Neural mechanisms generating the leech swimming rhythm: Swim-initator neurons excite the network of swim oscillator neurons. J. Comp. Physiol A. 161: 355-366.

This is a good set of ephys recordings. Except, here, 60 and 208 are opposite phases, which is not what I have them labelled as. This is the only 60 that I could find, but maybe this 60 is what I call 57 or 58. 

Nusbaum, M.P. (1986) Synaptic basis of swim initiation in the leech III. Synaptic effects of serotonin-containing interneurons (cells 21 and 61) on swim CPG neurones (cells 18 and 208). J. exp. Biol. 122:303-321.

Lockery, S.R., Kristan, W.B. (1990) Distributed Processing of Sensory Information in the Leech. II. Identification of Interneurons Contributing to the Local Bending Reflex. J. Neurosci. 10(6): 1816-1829.

This has 161, which is a P cell follower and matches my data. As well as 212, which I think I will call FCL 212.

So he reports that 161 only gets dorsal ipsi input, whereas 162 and 212 get broader input. This could possibly mean that 161 should be 162.

Perhaps also 159, 169, 157 he reports could correspond to 153, 155, 151 since these are also LB followers.

Baader, A.P. (1997) Interneuronal and motor patterns during crawling behavior of semi-intact leeches. J. Exp. Biol. 200: 1369-1381.

In this paper he labels some cells as cells I have identified, but this is all crawling so it is hard to know for sure if they match.

Wessel, R., Kristan Jr., W.B., Kleinfeld, D. (1999) Supralinear Summation of Synaptic inputs by an Invertebrate Neuron: Dendritic Gain Is Mediated by an "Inward Rectifier" K Current. J. Neurosci. 19(14):5875-5888.

Wessel, R., Kristan Jr., W.B., Kleinfeld, D. (1999) Dendrtic Ca-Activated K Conductances Regulate Eelctrical Signal Propagation in an Invertebrate Neuron. J. Neurosci. 19(19):8319-8326

These Wessel papers describe the AP cell. Can cite them as part of "AP cell has unknown function".

Ok, so I think there are a few changes to my table based on this review.

• My 60 to 61
• 61 is in phase with 208 and so my 60 is red so this is a switch
• 61 in Nusbaum, 1986
• My 58 to 60
• 60 is anti-phase with 208, which is cyan/green so this could be 57 or 58
• 60 in Nusbaum, 1987
• My 257 to 255
• Got these backwards from Kevin's paper. 255 is the swim oscillator, 257 does not oscillate.
• Briggman, 2006
• 205? Not sure what to do for this one
• Friesen website potential conflicts/matches
• 153 = 180 oscillation so that matches
• 152 = CM excitor during crawling
• 151 = NS
• 154 = photoreception
• 161 = EPSP from P cell
• 212 = EPSP from P cell (perhaps this should be FCL)

140319 - Coupling

Ok from a couple of swim bouts we can identify a bunch of swim neurons.

Then can target and record during swimming with ephys

Here is 208, colored by its phase:

I gave 208 a little stim near the 45s mark, but didn't seem to effect swim pattern. The DP nerve being shown is not from the same ganglion, but the phases are corrected.

And can target some other cells, 152, and 154:

Top - electrodes are recording from 152 and 154. Bottom, ephys of 152 and 154 shows oscillations during swim.

Here is the contralateral 154, and 153:

(This is much later in the experiment. The ganglion is dying and the VSD signals are getting weaker. The 153 oscillation is typically much much larger.)

Now we can excite 152 and see its functional coupling:

A. 152 (white) and 154 (cyan) are targeted with ephys. 154 is excited with 0.5 nA of current in a 1Hz square wave. 154, CV and 208 show inhibitory (out of phase) responses to 152 excitation. B. The coherence of voltage signals computed at 1Hz frequency. Red means in phase, cyan means out of phase. The colors are mapped onto components and shown on the ganglion in A. C. VSD traces of cells with significant coherence to 152. D. Ephys traces of 152 (red) and 154. (Bridge is over-compensated for 152, so what appears as down is actually up). Inhibitory response of 154 is seen in ephys trace. Dp nerve gets excited from 152 excitation, but no clear behavioral response.

Then we can inhibit the cell and see inhibitory functional coupling:

 A. 152 is inhibited. B. Coherence of VSD signals with 152. C. Traces of cells with sig coherence. D. Ephys of 152 and 154, no clear effect on 154 from inhibtion. Not much happens to DP. 

Oddly, with this it appears that the contralateral CV follows the inhibition in phase, while the ipsilateral CV follows out of phase. Perhaps 152 is electrically coupled to contra CV, and inhibits both CV.

Next, target 154. I ended up getting both bilateral pairs in this experiment.

So the SNR is starting to die, but I think its clear that 154 are connected bilaterally, and they both excite CV. Interestingly, 154 and CV are both in the purple colored phase of swim oscillation, while 152 is blue.

I also tried 153, but by the end the SNR of the VSD was so low that I wasn't confident about anything.

Here's the summary of 152 and 154 from this experiment:

140318 - fast network and s cell

Found the S-cell with an electrode and I wanted to check its coupling. I injected 0.4 nA of current at 0.5 Hz and then measured the followers of the S cells. I did the analysis with ICP and made the functional maps based on the coherence between the S cell voltage signal (black traces, cell body highlighted in white), and this was done with different settings.

What stands out as interesting is that the S-cell is strongly coupled to the AP cell, further suggesting that the S-cell and AP cell are part of the fast network. It is also connected to Accessory AP, and potentially another cell in the central packet thats part of the fast network. The green cells that get hyperpolarized seem real, less confident about the blue cells. The top trace uses 50 pcs, the bottom trace uses 130 (I think, maybe 100 or 150). The frequencies examined are from 0.5-0.6 ish.

This data is from x140316-t8.

ImageComponentParser: Trial Alignment

Performing the ICA analysis across multiple trials has some issues because of the random nature of the ICA decomposition. Running the analysis, even on the same data, does not mean that the same components will be the same source. One solution to this problem is to concatenate the trials, and perform ICA on the trials as if they were a single acquisition. In order for this to work, however, the pixels must stay aligned with the sources of the signals. The ImageComponentParser includes an image registration algorithm that can account for shifts in the imaging data across trials, which is a necessary step to perform concatenated trial ICA (ctICA). 

This registration algorithm can only perform an affine transformation on the entire image. If there is motion of a cell or a sub-region that is different from the macro-motion, then the algorithm cannot compensate for these changes. This will hinder the results of ctICA, but it can still give some good results as long as most pixels over a source remain over the source. 

The left is the raw image data, and the right is the data after each trial was aligned using the ImageComponentParser. The algorithm that is built into ICP only does full frame affine motion correction. This means that if a cell were to move relative to the ganglion, it would not be aligned as it should. Some better motion correction that can fix sub-regions of the image would improve the results, but this does a fairly good job. ICP automatically removes the edge pixels that are missing in some trials after the motion correction, giving only the region of the data that is present in all trials.

ImageComponentParser: Visual Intro

The ImageComponentParser is a Matlab GUI for automated analysis of functional imaging data (download page). It contains tools for exploring, extracting, and analyzing Calcium or VSD Imaging data. The primary tool of the package is to use ICA to automatically extract signals of interest from the data (papers on the technique: Mukamel et al. 2009 (post); Hill et al. 2010 (post)). The analysis suite consists of several stages, where the user can browse results and alter settings for each stage.

To get a sense for the challenges posed by imaging data and the utility of these automated methods here are some gifs that illustrate the analysis of optical data using this tool. The data presented in this post is calcium imaging data taken from Takaki Komiyama's lab from Akinori Mitani.

So making sense of data like this is quite a challenge. The standard method of extracting signals is to manually draw ROIs (Regions of Interest) around each cell. This is tedious and can miss signals. Further, some somas are not even in focus and the cells can be missed. PCA and ICA are automated techniques which can greatly aid in the extraction of the optical signals.

PCA looks at the correlations in the data and shows ensembles of cells which share certain activity patterns. ICP can be used to browse the PCs:

ICA looks for signals that are maximally independent. This can be used to extract the signals coming from individual cells.

Part of what makes this analysis useful is that no manual ROIs have to be drawn to extract cellular signals. This is especially useful for cells whose somas are not in focus, but have large axons/dendrites in the image. It would be a challenge just to draw an ROI around such cells, but the ICA algorithm can pull them out automatically.

ICA can further isolate noise components from signal components. However, ICA is naive to the difference between cellular signals and components from noise. The results must be filtered manually by hand, or simple algorithms could be used to pull out desired signals from noise signals (i.e. looking for spatially localized components). Here is an example of a noise component from this dataset:

The diagonal corrigated lines is likely an artifact from the two-photon acquisition (or possibly because of a motion-correction algorithm). You can see it pop-up for a few frames in the raw data. ICA pulls this noise out and it can be separated from the signals of interest.

Cincy

Major breakthrough achieved during my trip to Cincinnati -- We were able to record from both sides of the leech ganglion with voltage-sensitive dyes! This means that we can see virtually every cell in the ganglion all at once.


Here the P cell we stimulated is circled in red, with its VSD trace shown in blue (and red) and the intracellular shown in black. You can see other cells on the same side respond.


But here's the great thing, now we can see cells on the bottom, and a clear signal from one of the cells on the opposite side. We are ever closer from recording voltage from every neuron in this system!

Parental olfactory experience influences behavior and neural structure in subsequent generations

Dias, B.G., Ressler, K.J. (2013). Parental olfactory experience influences behavior and neural structure in subsequent generations. Nature Neuroscience.

The phenomenon of "inheritance of parental traumatic exposure".

We've long recognized that the ability to transmit information to offspring about specific environmental features or events would be remarkably advantageous. However, we've only been able to conceive of such "non-Mendelian" modes of inheritance only recently. Since Darwin, we have suspected that the information shared across generations was only crafted by natural selection and that traits were determined by who best reproduced and not through the experiences of the orgnaisms. But new empirical data is showing that epigentics can act as the mechanism to pass down information learned from the environment to the next generation.

A standard behavioral test is called Fear conditioning -- an animal is taught to associate an arbitrary stimulus was some kind of painful shock. Animals learn these associations very quickly, and we can measure an animal's startle response to the arbitrary stimulus as an indicator of how well an animal learned to associate the stimulus with the shock.

Three generation of mice were studied -- F0 were the grandparents and put in 3 conditions: no smells, exposed to acetophenone (smells like cherries), and exposed to propanol (control smell). The offspring of F0 (F1) were then tested to see if they responded to the smell that their parents were conditioned to. The mice responded more to the odor that their parents were conditioned to, but not the other odor, even though these mice were never exposed to either odors. This even extended to the next generation (F2).

Further, F1 mice showed higher sensitivity to the smell that their parents were conditioned with.

Cross-fostering experiments indicated that the behavioral changes were not passed down from social learning and indicated that the enhanced sensitivity and behavioral changes were genetic.

The glomeruli (packet of neurons that respond to specific smells) that respond to acetophenone showed anatomical differences when the parents were conditioned to acetophenone.

Figure 4 Behavioral sensitivity and neuroanatomical changes are inherited in F2 and IVF-derived generations. (a,b) Responses of F2-C57Bl/6J males revealed that F2-Ace-C57 mice had an enhanced sensitivity to acetophenone compared with F2-Prop-C57 mice (a). In contrast, F2-Prop-C57 mice had an enhanced sensitivity to propanol compared with F2-Ace-C57 mice (b; F2-Prop-C57, n = 8; F2-Ace-C57, n = 12; OPS to acetophenone: t test, P = 0.0158, t18 = 2.664; OPS to propanol: t test, P = 0.0343, t17 = 2.302). (c–f). F2-Ace-M71 mice whose F0 generation male had been conditioned to acetophenone had larger dorsal and medial M71 glomeruli in the olfactory bulb than F2-Prop-M71 mice whose F0 generation had been conditioned to propanol. Scale bar represents 200 µm. (g) Dorsal M71 glomerular area in F2 generation (M71-LacZ: F2-Prop, n = 7; F2-Ace, n = 8; t test, P < 0.0001, t13 = 5.926). (h) Medial M71 glomerular area in F2 generation (M71-LacZ: F2-Prop, n = 6; F2-Ace, n = 10; t test, P = 0.0006, t14 = 4.44). (i) Dorsal M71 glomerular area in IVF offspring (F1-Prop-IVF, n = 23; F1-Ace-IVF, n = 16; t test, P < 0.001, t37 = 4.083). (j) Medial M71 glomerular area in IVF offspring (F1-Prop-IVF, n = 16; F1-Ace-IVF, n = 19; t test, P < 0.001, t33 = 5.880). Data are presented as mean ± s.e.m.*P < 0.05, **P < 0.01, ***P < 0.001, ****P < 0.0001.

Finally, they looked at the methylation patterns on the odorant receptor genes in the sperm of the F0 mice. Methylation has been recognized as a potential marker for epi-genetic changes in transcription. There was a significant decrease in the amount of methylation of Olfr151 (which is responsive to acetophone), but no changes in Olfr6 (which is not responsive).

The idea that the experience of ones ancestors can influence behavior has important consequences. This may contribute to an "intergenerational transmission of risk for neurophsychiatric disorders, such as phobias, anxiety and post-traumatic stress disorder".

The fact that changes can propagate over more than one generation has implications for the process of evolution -- the actual experience of the parents altering the genetic makeup of the offspring means we may have to refine the way we think about how evolution works. However, it is unclear if these changes can last for a long enough time (over many generations) to truly alter evolution, as methylation patterns may not be stable enough for natural selection to have a long-term influence. But there could be mechanisms that can incorporate epi-genetic changes into the genome.

ICA Swim Maps

Neurons with coherent oscillations to the swim motor pattern colored in by phase.

network, milliseconds

The current state-of-the-art AI is based on "deep belief networks". These are neural models based on brain inpsired ideas, and are now beginning to be used effectively for AI. Part of the reason seems to be just some learning tricks ( like not minimizing the error function completely), and part is that computing power has become sufficient to make large and deep networks.

One of the big components is the "max pooling", which is modeled off of complex cells in visual cortex. The idea is that the simple cells respond to a convolutional filter -- like the gabor set of V1, or the ICs. And the complex cells listen to a range of simple cells and respond as the maximum of their inputs.

Then they build hierarchies of these networks. Convolutional filter, max pooling, convolutional filter max pooling. This is building a hiearchical component decomposition of the stimulus, which allows for local invariances of the features at different feature levels. With these types of networks they can now achieve human level performance on certain types of visual recognition tasks.

These advances are brilliant and I think are very close to part of the way the brain works. These type of deep belief networks don't have any feedback loops, but clearly the brain is full of them. What these networks seem to be is the "feed-forward" pathway of the cortical hierarchy. Well not really the cortical hiearchy, but a hierarchy that they learned via some derived learning rule. Essentially, their machine is like a human that is really well trained at recognizing a set of objects. Except the task isn't like a human staring at a picture, rather it is like the image flashes on the screen for 100 ms and you have to respond as fast as possible. I imagine with a lot of training a human could do this task as effectively as the deep belief nets.

I think this hits on an essential aspect of what the cortical hierarchy is doing. It is forming a component decomposition of the incoming sensory space. Now, the complex cells may be doing something kind of like max-pooling, but there could be much more sophisticated ways of pooling. If we imagine the visual hierarchy then thalamic inputs are sending in sensory signals into L4 of V1. Here, the basic "pixels" of the sensory world are being represented in a overcomplete feature space. This is analogous to the first simple cell layer of these deep belief networks.

In cortex, L4 seems to mainly project to L2/3. L2/3 characterized by many recurrent connections, it is close to the top-down input, and thought to be where the complex cells are. I'm not a fan of max pooling as what L2/3 is doing per se. I think that max pooling is just something that seems to happen because of both the way we study it and because it is doing something a little bit related. I feel like it is the "model" of what you see. L2/3 is like what you see when you imagine seeing it. So when you actually see something both L4 and L2/3 are going to be active -- as you are simultaneously seeing and classifying/modeling what you see. L2/3 seems to be just locally max-pooling, but it is probably just less sensitive to the local features and pooling data from more distal sources.

So the deep belief networks model the feed-forward pathway. Low-level features in V1, texture features in V2, shape features in V4, motion in MT. The higher level areas were likely shaped by evolution to be particularly good at making their transformations. The nervous system is so amazingly flexible, that the mechanisms for the computations up the feed-forward pathway could be very diverse. But the main idea would be looking for particular correlations in the features.

The brain seems to not be a simple vertical hierarchy as many of the deep belief nets work, but it kind of fans out and then comes back together at the top. Further, the brain utilizes every level of the feature space -- whereas these deep belief nets seem to just rely on the output of the top (mainly because they are trying to make a classification). There is the what and the where pathway -- an object-centric spatially invariant classification system, and a relational, predictive, operational centers bound together at various levels in the hierarchy.

The what pathway is like the object consciousness. When you look at anything, say my dog suki, your brain is representing information across the cortex simultaneously. V1 is representing the low-level sensory information -- the color of suki's hair, her outline, the slits in her close eyes. V2 is unifying the hair as a contiguous texture, V4 is outlining her dimensions creating a 3D model of her shape that you can just feel. MT picks up on the subtle motions of her breathe, and notices her ears flicker.

Further down the what pathway more "higer-level" features seem to be represented. Face patches respond to different types of invariances -- regions for profiles, regions for frontal face. My what pathway connects me to the knowledge that I'm looking at a dog, dogs have these typical shape. Higher up I know that this is a particular dog. Higher up I associated my daily memories with suki. Every time I look at suki this information cascades up in a few 100 milliseconds and becomes accessible to the rest of my brain.

Jonathan Shlens

he came from google to give a talk about using semantic context to improve visual systems. It was quite good.

Basically he combined a deep learning visual network, with one of these semantic networks. He made a mapping between the visual network and the semantic network that put visual cues into a continuous semantic space. Then he could teach the visual network these image classes (hand labeled from imageNET(?)) and it would learn the mapping between the visual cues and the semantic space. The semantic space was learned from wikipedia -- just like next word predictors (that are also deep I think). The visual network could then recognize completely new visual categories (not just new images within a learned category). Because the semantic space would link the categories (the category relationships are learned through wikipedia). He makes the metaphor that the visual system has seen a desk chair, but never a rocking chair. But the semantic system has read about a rocking chair and its relation to a desk chair and other chairs. The system then sees a rocking chair for the first time and can infer that it is indeed a rocking chair through its semantic knowledge.

There is also this trick of using basically a hash-table to quickly get out approximate filters in a deep network, this speeds up the system by almost 20,000x.
Here is the paper

I can't find much of his other papers. He was in EJ's lab and has a lot of papers with Greg fields and Jeff Gauthier.

And there is this -- the code for a rbm/deep belief net run on gpu
https://code.google.com/p/cuda-convnet/

And this which was used to get all the stuff from wikipedia:
https://code.google.com/p/word2vec/

Kernel ICA is too slow

yeah, looks like it does effective ICA, but it takes forever. 8 components takes 2 minutes, 16 takes 10 minutes, 32 takes almost an hour.

I'm running it up to 128, so if it keeps going up in the same pattern -- 64 will take 5 hours, 128 will take more than a day...

Schmidhuber Deep Learning Review

http://www.idsia.ch/~juergen/firstdeeplearner.html

A good set of articles and timeline about deep learning