Search
Now showing items 1-2 of 2
Streaming Normalization: Towards Simpler and More Biologically-plausible Normalizations for Online and Recurrent Learning
(Center for Brains, Minds and Machines (CBMM), arXiv, 2016-10-19)
We systematically explored a spectrum of normalization algorithms related to Batch Normalization (BN) and propose a generalized formulation that simultaneously solves two major limitations of BN: (1) online learning and ...
How Important is Weight Symmetry in Backpropagation?
(Center for Brains, Minds and Machines (CBMM), arXiv, 2015-11-29)
Gradient backpropagation (BP) requires symmetric feedforward and feedback connections—the same weights must be used for forward and backward passes. This “weight transport problem” [1] is thought to be one of the main ...