I've been thinking about this some and in terms of a feedback loop it may not be necessary to do the sqrt part. If the interneurons were doing the L2 norm correctly in a feedback loop, then it should always settle to an equilibrium value where the length of the vector is 1. This will be true no matter how large the inputs are, and it should be the only stable state. But if the sqrt is removed, then the normalization should be over-compensated when the vector length is above 1 and under-compensated when the vector length is below 1. This still sends the dynamics to the same stable state, though, it would just probably over/under shoot, but still settle to the correct length.
Without doing the square summation the feedback normalization was not strong enough, I guess, to compensate for the feedback excitation. I think its just that the feedback doesn't grow enough compared to the excitation, so it cannot keep the population normalized to a single value. But if the feedback grows too much, it would just reduce the activity too much and then it would settle to some eq.
...yeah of course not. I'm actually less sure if its even possible regardless if the sqrt would work -- at least from a purely feedback stance. If there's more FF, then there would have to be more inhibition to compensate, but if it always decayed to a stable integral, then the inhibition level would have to be the same. If the inhibition was the same, it couldn't possibly reduce the interneurons enough. So there can't be a feedback circuit that always maintains a normalized vector -- unless you add in feed-forward inhibition too.
It does a pretty good job of keeping it under control in general. I don't have to fiddle with the knobs too much. The feedback has some interesting properties.
No comments:
Post a Comment