Search
Now showing items 1-4 of 4
System identification of neural systems: If we got it right, would we know?
(Center for Brains, Minds and Machines (CBMM), 2022-07-02)
Various artificial neural networks developed by engineers have been evaluated as models of the brain, such as the ventral stream in the primate visual cortex. After being trained on large datasets, the network outputs are ...
PCA as a defense against some adversaries
(Center for Brains, Minds and Machines (CBMM), 2022-03-30)
Neural network classifiers are known to be highly vulnerable to adversarial perturbations in their inputs. Under the hypothesis that adversarial examples lie outside of the sub-manifold of natural images, previous work has ...
SGD Noise and Implicit Low-Rank Bias in Deep Neural Networks
(Center for Brains, Minds and Machines (CBMM), 2022-03-28)
We analyze deep ReLU neural networks trained with mini-batch stochastic gradient decent and weight decay. We prove that the source of the SGD noise is an implicit low rank constraint across all of the weight matrices within ...
Compositional Sparsity: a framework for ML
(Center for Brains, Minds and Machines (CBMM), 2022-10-10)
The main claim of this perspective is that compositional sparsity of the target function, which corre- sponds to the task to be learned, is the key principle underlying machine learning. I prove that under restrictions of ...