Disentangling with Biological Constraints: A Theory of Functional Cell Types
Neural representations are often highly sought after in both brains and machine learning.
We demonstrate these constraints lead to disentangling in a variety of tasks and architectures, including variational autoencoders.
We also use this theory to explain why the brain partitions its cells into distinct cell types such as grid and object-vector cells, and also explain when the brain instead entangles representations in response to entangled task factors.
Authors
James C.R. Whittington, Will Dorrell, Surya Ganguli, Timothy E.J. Behrens