nLab machine learning

Idea

Machine learning is a branch of computer science which devises algorithms to learn from data so as to perform tasks without being explicitly programmed to do so. Notable approaches include neural networks, support vector machines and Bayesian networks.

Related concepts

• neural network

• kernel method

• topological data analysis

• topological machine learning

• statistical learning theory

• singular learning theory

• information geometry

References

Classical machine learning

Textbook accounts:

• Sumio Watanabe, Algebraic geometry and statistical learning theory, CRC Press (2009) [doi:10.1017/CBO9780511800474]

• Sumio Watanabe, Mathematical theory of Bayesian statistics, Cambridge University Press (2018) [ISBN:9780367734817, pdf]

• Shai Shalev-Shwartz, Shai Ben-David, Understanding machine learning: from theory to algorithms, Cambridge University Press (2014) [doi:10.1017/CBO9781107298019]

• Ian Goodfellow, Y. Bengio, A. Courville, Deep learning, MIT Press (2016) [web, pdf, ISBN:9780262035613]

See also:

• Wikipedia, Machine learning
• Juergen Schmidhuber (2015), Scholarpedia, 10(11):32832 Deep learning (with extensive historical bibliography)

Expository review:

• Stephen Wolfram, What Is ChatGPT Doing… and Why Does It Work? (Feb 2023)

On transformers and large language models (LLM):

• Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Łukasz Kaiser, Illia Polosukhin, Attention is all you need, in Advances in Neural Information Processing Systems 30 (NIPS 2017) pdf

• Michael R. Douglas, Large language models, arXiv:2307.05782

With regards to kernel methods:

• Thomas Hofmann, Bernhard Schölkopf, Alexander J. Smola, Kernel methods in machine learning, Annals of Statistics 2008, Vol. 36, No. 3, 1171-1220 (arXiv:math/0701907)

• Julien Mairal, Jean-Philippe Vert, Machine Learning with Kernel Methods, 2017 (pdf, pdf)

• Hông Vân Lê, Supervised learning with probabilistic morphisms and kernel mean embeddings, arXiv:2305.06348

Proposal for applications of category theory to machine learning:

• Dan Shiebler, Bruno Gavranović, Paul Wilson, Category Theory in Machine Learning (arXiv:2106.07032)
• Yiannis Vlassopoulos, Modelling language semantics and probability densities of text continuations, QNLP 2022 youtube

The following framework claims to relate to homotopy type theory

• Ben Goertzel, Reflective metagraph rewriting as a foundation for an AGI “Language of Thought”, arXiv:2112.08272

On a definition of artificial general intelligence

• S. Legg, M. Hutter, Universal intelligence: a definition of machine intelligence, Minds & Machines 17, 391–444 (2007) doi

• M. Hutter, Universal artificial intelligence: sequential decisions based on algorithmic probability, Springer 2005; book presentation pdf

• Shane Legg, Machine super intelligence, PhD thesis, 2008 pdf

Quantum machine learning

In view of quantum computation (ie. quantum machine learning):

• Diego Ristè, Marcus P. da Silva, Colm A. Ryan, Andrew W. Cross, Antonio D. Córcoles, John A. Smolin, Jay M. Gambetta, Jerry M. Chow & Blake R. Johnson,

  Demonstration of quantum advantage in machine learning, npj Quantum Information volume 3, Article number: 16 (2017) (doi:10.1038/s41534-017-0017-3)

• Jacob Biamonte, Peter Wittek, Nicola Pancotti, Patrick Rebentrost, Nathan Wiebe, Seth Lloyd, Quantum Machine Learning, Nature 549 (2017) 195–202 (doi:10.1038/nature23474)

  EurekaAlert, Quantum Machine Learning 14-Sep-2017

• Iris Cong, Soonwon Choi, Mikhail D. Lukin, Quantum convolutional neural networks, Nature Physics volume 15, pages 1273–1278 (2019) (doi:10.1038/s41567-019-0648-8)

  (on quantum neural networks)

• Yunchao Liu, Srinivasan Arunachalam, Kristan Temme, A rigorous and robust quantum speed-up in supervised machine learning (arXiv:2010.02174)

• Melanie Swan, Renato P dos Santos, Frank Witte, Between Science and Economics, Volume 2: Quantum Computing Physics, Blockchains, and Deep Learning Smart Networks, World Scientific 2020 (doi:10.1142/q0243)

• Stefano Mangini, Francesco Tacchino, Dario Gerace, Daniele Bajoni, Chiara Macchiavello, Quantum computing models for artificial neural networks, EPL (Europhysics Letters) 134(1), 10002 (2021) (arXiv:2102.03879)

review:

• Kamila Zaman, Alberto Marchisio, Muhammad Abdullah Hanif, Muhammad Shafique, A Survey on Quantum Machine Learning: Current Trends, Challenges, Opportunities, and the Road Ahead [arXiv:2310.10315]

and with emphasis on classically controlled NISQ-computes:

• John Preskill, Section 6.5 of: Quantum Computing in the NISQ era and beyond, Quantum 2018-08-06, volume 2, page 79 (arXiv:1801.00862, doi:10.22331/q-2018-08-06-79)

• Kosuke Mitarai, Makoto Negoro, Masahiro Kitagawa, Keisuke Fujii, Quantum Circuit Learning, Phys. Rev. A 98, 032309 (2018) (arXiv:1803.00745)

• Marcello Benedetti, Erika Lloyd, Stefan Sack, Mattia Fiorentini, Parameterized quantum circuits as machine learning models, Quantum Science and Technology 4, 043001 (2019) (arXiv:1906.07682)

• Marco Radic, Quantum-enhanced Machine Learning in the NISQ era, 2019 (pdf, pdf)

• Dario Gerace, Quantum Machine Learning on NISQ hardware,, talk at INT online program on “Scientific Quantum Computing and Simulation on Near-Term Devices” 2020 (pdf, pdf)

• Andrea Mari, Thomas R. Bromley, Josh Izaac, Maria Schuld, Nathan Killoran, Transfer learning in hybrid classical-quantum neural networks, Quantum 4, 340 (2020) (arXiv:1912.08278)

• Lucas Munds, Quantum Machine Learning: A Roadmap for NISQ Era and Beyond, Oct 17, 2020

• Tirthajyoti Sarkar, TensorFlow Quantum: Marrying machine learning with quantum computing, Mar 11, 2020

• QunaSys, Tech Blog, Quantum machine learning of quantum data with NISQ devices (Feb 15 2021)

• TensorFlow.org (Google), Quantum Machine Learning

Critical discussion:

• Pablo Bermejo, Paolo Braccia, Manuel S. Rudolph, Zoë Holmes, Lukasz Cincio, M. Cerezo: Quantum Convolutional Neural Networks are (Effectively) Classically Simulable [arXiv:2408.12739]

  “We present this as a challenge to the community and argue that until the viability of QCNNs for [not classically simulable] datasets can be demonstrated there is no reason to believe QCNNs will be useful. […] Hence we boldly claim: There is currently no evidence that QCNNs will work on classically non-trivial tasks, and their place in the upper echelon of promising QML architectures should be seriously revised.”

Applications

There are or will be innumerable applications. Here are some:

To mathematical structures in algebraic geometry, representation theory, number theory and combinatorics:

• Yang-Hui He, Machine-Learning Mathematical Structures, International Journal of Data Science in the Mathematical Sciences [arXiv:2101.06317, doi:10.1142/S2810939222500010]

reviewed in:

• Yang-Hui He, Machine-Learning Mathematical Structures, talk at M-Theory and Mathematics 2023, NYU Abu Dhabi (Jan 2023) [web]

To the conformal bootstrap:

• Gergely Kántor, Vasilis Niarchos, Constantinos Papageorgakis, Solving conformal field theories with artificial intelligence (arXiv:2108.08859)

(…)