| dc.description.abstract | Sample-efficient learning and flexible generalization are hallmarks of intelligent behavior. Both sample-efficient learning and flexible generalization rely on re-using a mental model of the world in new contexts. For many decades, researchers in cognitive science, neuroscience, and machine learning have studied competing theories about the structure of our mental model of the world. One set of theories concerns the structure of multi-object representations in the brain. Some studies claim the brain represents multiple objects by allocating them to disjoint “slots” in working memory, others claim that the brain flexibly distributes a common pool of resources across objects, and yet others claim the brain represents multiple objects by rapidly switching between them through time. Another set of theories concerns the nature of predicting object motion. Some claim that the mind has an internal model of physics in the world that it uses to simulate the motion of objects through time, whereas others claim the mind relies on priors and heuristics to predict object motion without explicit simulation. Both of these sets of competing theories are long-standing and unresolved. In this work, we tackle these two open questions using primate neurophysiology and computational modeling. We trained monkeys to perform multi-object memory and motion prediction tasks, recorded large-scale single-unit activity from frontal cortex brain areas, and rigorously compared different hypotheses for the neural mechanisms of multi-object working memory and motion prediction. In the case of multi-object working memory, we found that the neural activity we recorded is more consistent with a model that flexibly distributes attentional resources across objects than with models that use object slots or temporal switching representations. In the case of motion prediction, we found that the neural activity is not consistent with the monkeys simulating an occluded moving object in real-time. Instead, the monkeys’ neural activity is driven largely by an anticipation of the position of the object at a future point in time. Both of these findings call into question long-standing cognitive theories and imply that the brain’s model of the world incorporates attentional mechanisms, priors, and heuristics. Lastly, we introduce a neural data preprocessing method for stabilizing electrophysiology recordings. This method improves spike-sorting results, helped us recover more neurons from our data, and we hope may help others make the most of their electrophysiology data as well. | |
