Show simple item record

dc.contributor.advisorTomaso Poggio
dc.contributor.authorBouvrie, Jakeen_US
dc.contributor.authorPoggio, Tomasoen_US
dc.contributor.authorRosasco, Lorenzoen_US
dc.contributor.authorSmale, Steveen_US
dc.contributor.authorWibisono, Andreen_US
dc.contributor.otherCenter for Biological and Computational Learning (CBCL)en_US
dc.date.accessioned2010-11-22T22:15:09Z
dc.date.available2010-11-22T22:15:09Z
dc.date.issued2010-11-19
dc.identifier.urihttp://hdl.handle.net/1721.1/60024
dc.description.abstractHierarchical learning algorithms have enjoyed tremendous growth in recent years, with many new algorithms being proposed and applied to a wide range of applications. However, despite the apparent success of hierarchical algorithms in practice, the theory of hierarchical architectures remains at an early stage. In this paper we study the theoretical properties of hierarchical algorithms from a mathematical perspective. Our work is based on the framework of hierarchical architectures introduced by Smale et al. in the paper "Mathematics of the Neural Response", Foundations of Computational Mathematics, 2010. We propose a generalized definition of the neural response and derived kernel that allows us to integrate some of the existing hierarchical algorithms in practice into our framework. We then use this generalized definition to analyze the theoretical properties of hierarchical architectures. Our analysis focuses on three particular aspects of the hierarchy. First, we show that a wide class of architectures suffers from range compression; essentially, the derived kernel becomes increasingly saturated at each layer. Second, we show that the complexity of a linear architecture is constrained by the complexity of the first layer, and in some cases the architecture collapses into a single-layer linear computation. Finally, we characterize the discrimination and invariance properties of the derived kernel in the case when the input data are one-dimensional strings. We believe that these theoretical results will provide a useful foundation for guiding future developments within the theory of hierarchical algorithms.en_US
dc.format.extent59 p.en_US
dc.relation.ispartofseriesMIT-CSAIL-TR-2010-051en_US
dc.relation.ispartofseriesCBCL-292en_US
dc.rightsCreative Commons Attribution-NonCommercial-NoDerivs 3.0 Unporteden
dc.rights.urihttp://creativecommons.org/licenses/by-nc-nd/3.0/
dc.subjecthierarchical learningen_US
dc.subjectkernel methodsen_US
dc.subjectlearning theoryen_US
dc.titleGeneralization and Properties of the Neural Responseen_US


Files in this item

Thumbnail
Thumbnail

This item appears in the following Collection(s)

Show simple item record