CBMM Memo Series: Recent submissions

Now showing items 10-12 of 147

    • Skip Connections Increase the Capacity of Associative Memories in Variable Binding Mechanisms 

      Xie, Yi; Li, Yichen; Rangamani, Akshay (Center for Brains, Minds and Machines (CBMM), 2023-06-27)
      The flexibility of intelligent behavior is fundamentally attributed to the ability to separate and assign structural information from content in sensory inputs. Variable binding is the atomic computation that underlies ...

    • Feature learning in deep classifiers through Intermediate Neural Collapse 

      Rangamani, Akshay; Lindegaard, Marius; Galanti, Tomer; Poggio, Tomaso (Center for Brains, Minds and Machines (CBMM), 2023-02-27)
      In this paper, we conduct an empirical study of the feature learning process in deep classifiers. Recent research has identified a training phenomenon called Neural Collapse (NC), in which the top-layer feature embeddings ...

    • SGD and Weight Decay Provably Induce a Low-Rank Bias in Deep Neural Networks 

      Galanti, Tomer; Siegel, Zachary; Gupte, Aparna; Poggio, Tomaso (Center for Brains, Minds and Machines (CBMM), 2023-02-14)
      In this paper, we study the bias of Stochastic Gradient Descent (SGD) to learn low-rank weight matrices when training deep ReLU neural networks. Our results show that training neural networks with mini-batch SGD and weight ...