In the context of machine learning, singular learning theory applies results from algebraic geometry to statistical learning theory. In the case of learning algorithms, such as deep neural networks, where there are multiple parameter values corresponding to the same statistical distribution, the preimage of the target distribution may take the form of a singular subspace of the parameter space. Techniques from algebraic geometry may then be applied to study learning with such devices.
Historically, it has been understood that the neural networks are singuar statistical models in
Textbook treatments:
Sumio Watanabe, Algebraic geometry and statistical learning theory, CRC Press (2009) [doi:10.1017/CBO9780511800474]
Sumio Watanabe, Mathematical theory of Bayesian statistics, Cambridge University Press (2018) [ISBN:9780367734817, pdf]
For an informal discussion:
For a series of talks and further texts:
Singular Learning Theory seminar, (webpage)
S. Wei, Daniel Murfet, M. Gong, H. Li , J. Gell-Redman, T. Quella, Deep learning is singular, and that’s good, IEEE Transactions on neural networks and learning systems pdf
Last revised on April 7, 2023 at 12:33:31. See the history of this page for a list of all contributions to it.