What is Cortex?

When we look at the brain, especially the human brain, what we see is something people call "isocortex". Isocortex became the name for cortex because it essentially looks the same all throughout the brain. What is remarkable about cortex, however, is the vast diversity of the functions it implements. Cortex handles all of the sensory processing - vision, audition, somatosensory, taste, small, proprioception (where your muscles are), interoception (what the rest of your body is telling you). And cortex does all the high-level stuff - planning, speaking, thinking, decision-making.

Considering all of these functions and the general uniformity of Cortex, we can see that Cortex is general-purpose and extremely flexible. So what is it doing? What is the commmon functionality that could be used for so many diverse tasks? I think the answer is that Cortex is building a generative-model of its inputs.

A generative model is a system which can take in inputs, builds a parameterized model of the inputs, and reproduce the inputs from the parameters. The reproducing of the inputs is key, as we can use the differences between the true inputs and the reproduced inputs as part of a learning rule. This is very analogous to RBMs, as essentially RBMs build a parameter set and try to regenerate the inputs. The weights of the RBM are modified based on the differences between the true inputs and the generated inputs. Cortex is in many ways a much-more powerful generalization of RBMs.

Now, Cortex isn't exactly the same throughout the human brain. I'm currently very interested in the evolution of Cortex, as it appears throughout the course of evolution Cortex has been slightly modified, and specialized for its various tasks. I think a plausible story about how the brain was evolved was that a simple, very general Cortex emerged and was extremely useful. Quickly it expanded to handle processing of all types of sensory modalities and other functions. Evolution is very good at copying structures and utilizing them for other tasks. Once Cortex started being used for all of these different purposes, it started specializing. Evolution figured out tricks to make the different areas of Cortex more efficient - allowing for more cortex and more processing power. It also figured out ways to keep Cortex stable and ways to have Cortex build its generative model more quickly and efficiently. Consider V1 as the first feature level representation of the world. Neurons in V1 are typically known to be selective to orientations and edges - this is the first level of a generative model of a 3D world. There is evidence, however, that V1 can produce these features without the need of any inputs. I think that evolution has set the parameters of V1 such that its initial conditions are close to correct for making a 3D model, so this would make cortex look the same even without any input. However, it is likely that these neuron's feature selectivity is very unrefined, and V1 almost surely needs inputs to be as accurate as it is. Evolution has just figured out a way to get V1's initial conditions close to the ideal local-minimum, so that it quickly settles into the correct format and doesn't get stuck in a different local-minimum.