Browsing CBMM Memo Series by Author "Rangamani, Akshay"

Now showing items 1-5 of 5

    • Feature learning in deep classifiers through Intermediate Neural Collapse 

      Rangamani, Akshay; Lindegaard, Marius; Galanti, Tomer; Poggio, Tomaso (Center for Brains, Minds and Machines (CBMM), 2023-02-27)
      In this paper, we conduct an empirical study of the feature learning process in deep classifiers. Recent research has identified a training phenomenon called Neural Collapse (NC), in which the top-layer feature embeddings ...

    • For interpolating kernel machines, the minimum norm ERM solution is the most stable 

      Rangamani, Akshay; Rosasco, Lorenzo; Poggio, Tomaso (Center for Brains, Minds and Machines (CBMM), 2020-06-22)
      We study the average CVloo stability of kernel ridge-less regression and derive corresponding risk bounds. We show that the interpolating solution with minimum norm has the best CVloo stability, which in turn is controlled ...

    • The Janus effects of SGD vs GD: high noise and low rank 

      Xu, Mengjia; Galanti, Tomer; Rangamani, Akshay; Rosasco, Lorenzo; Poggio, Tomaso (2023-12-21)
      It was always obvious that SGD has higher fluctuations at convergence than GD. It has also been often reported that SGD in deep RELU networks has a low-rank bias in the weight matrices. A recent theoretical analysis linked ...

    • Skip Connections Increase the Capacity of Associative Memories in Variable Binding Mechanisms 

      Xie, Yi; Li, Yichen; Rangamani, Akshay (Center for Brains, Minds and Machines (CBMM), 2023-06-27)
      The flexibility of intelligent behavior is fundamentally attributed to the ability to separate and assign structural information from content in sensory inputs. Variable binding is the atomic computation that underlies ...

    • Understanding the Role of Recurrent Connections in Assembly Calculus 

      Rangamani, Akshay; Xie, Yi (Center for Brains, Minds and Machines (CBMM), 2022-07-06)
      In this note, we explore the role of recurrent connections in Assembly Calculus through a number of experiments conducted on models with and without recurrent connections. We observe that as- semblies can be formed even ...