A function machine is a generalization of neural networks to potentially infinite dimensional layers, motivated by the study of universal approximation of operators and functionals over abstract Banach spaces.
Definition
A function machine’s layers is defined by the type of layers it is composed with.
Definition
Let both compact, is some affine map.
(Operator layer) When , is said to be an operator layer if there is some continuous family of measures indexed by and a function such that:
(Functional layer) When , is said to be a functional layer if there is some measure and a vector such that:
(Basis layer) When , is said to be a basis layer if there is some function and such that:
(Fully-connected layer) When the inputs and outputs of a layer are finite dimensional, we yield the standard fully-connected layer, denoted
One can now, for instance, define an 1-hidden layer operator network to be the composition of two operator layers:
where , are families of absolutely continuous measures w.r.t. the Lebesgue measure , and are measurable bias functions over compact domains and respectively, and an activation function.
William Guss, Deep Function Machines: Generalized Neural Networks for Topological Layer Expression,(arXiv:1612.04799
William Guss, Ruslan Salakhutdinov, On Universal Approximation by Neural Networks with Uniform Guarantees on Approximation of Infinite Dimensional Maps,(arXiv:1910.01545
Last revised on January 14, 2020 at 22:15:44.
See the history of this page for a list of all contributions to it.