nLab machine learning

Idea

Machine learning is a branch of computer science which devises algorithms to learn from data so as to perform tasks without being explicitly programmed to do so. Notable approaches include neural networks, support vector machines and Bayesian networks.

Related concepts

• neural network

• kernel method

• topological data analysis

• topological machine learning

• statistical learning theory

• singular learning theory

• information geometry

References

Classical machine learning

Textbook accounts:

• Sumio Watanabe, Algebraic geometry and statistical learning theory, CRC Press (2009) [doi:10.1017/CBO9780511800474]

• Sumio Watanabe, Mathematical theory of Bayesian statistics, Cambridge University Press (2018) [ISBN:9780367734817, pdf]

• Shai Shalev-Shwartz, Shai Ben-David, Understanding machine learning: from theory to algorithms, Cambridge University Press (2014) [doi:10.1017/CBO9781107298019]

• Ian Goodfellow, Y. Bengio, A. Courville, Deep learning, MIT Press (2016) [web, pdf, ISBN:9780262035613]

See also:

• Wikipedia, Machine learning
• Juergen Schmidhuber (2015), Scholarpedia, 10(11):32832 Deep learning (with extensive historical bibliography)

Expository review:

• Stephen Wolfram, What Is ChatGPT Doing… and Why Does It Work? (Feb 2023)

On transformers and large language models (LLM):

• Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Łukasz Kaiser, Illia Polosukhin, Attention is all you need, in Advances in Neural Information Processing Systems 30 (NIPS 2017) pdf

• Michael R. Douglas, Large language models, arXiv:2307.05782

With regards to kernel methods:

• Thomas Hofmann, Bernhard Schölkopf, Alexander J. Smola, Kernel methods in machine learning, Annals of Statistics 2008, Vol. 36, No. 3, 1171-1220 (arXiv:math/0701907)

• Julien Mairal, Jean-Philippe Vert, Machine Learning with Kernel Methods, 2017 (pdf, pdf)

• Hông Vân Lê, Supervised learning with probabilistic morphisms and kernel mean embeddings, arXiv:2305.06348

Proposal for applications of category theory to machine learning:

• Dan Shiebler, Bruno Gavranović, Paul Wilson, Category Theory in Machine Learning (arXiv:2106.07032)
• Yiannis Vlassopoulos, Modelling language semantics and probability densities of text continuations, QNLP 2022 youtube

The following framework claims to relate to homotopy type theory

• Ben Goertzel, Reflective metagraph rewriting as a foundation for an AGI “Language of Thought”, arXiv:2112.08272

On a definition of artificial general intelligence

• S. Legg, M. Hutter, Universal intelligence: a definition of machine intelligence, Minds & Machines 17, 391–444 (2007) doi

• M. Hutter, Universal artificial intelligence: sequential decisions based on algorithmic probability, Springer 2005; book presentation pdf

• Shane Legg, Machine super intelligence, PhD thesis, 2008 pdf

Quantum machine learning

In view of quantum computation (ie. quantum machine learning):

• Srinivasan Arunachalam, Ronald de Wolf: A Survey of Quantum Learning Theory, ACM SIGACT News 48 2 (2017) 41-67 [arXiv:1701.06806, doi:10.1145/3106700.3106710]

• Blake R. Johnson et al.: Demonstration of quantum advantage in machine learning, npj Quantum Information volume 3, Article number: 16 (2017) (doi:10.1038/s41534-017-0017-3)

• Jacob Biamonte, Peter Wittek, Nicola Pancotti, Patrick Rebentrost, Nathan Wiebe, Seth Lloyd, Quantum Machine Learning, Nature 549 (2017) 195–202 (doi:10.1038/nature23474)

  EurekaAlert, Quantum Machine Learning 14-Sep-2017

• Iris Cong, Soonwon Choi, Mikhail D. Lukin, Quantum convolutional neural networks, Nature Physics volume 15, pages 1273–1278 (2019) (doi:10.1038/s41567-019-0648-8)

  (on quantum neural networks)

• Yunchao Liu, Srinivasan Arunachalam, Kristan Temme, A rigorous and robust quantum speed-up in supervised machine learning (arXiv:2010.02174)

• Melanie Swan, Renato P dos Santos, Frank Witte, Between Science and Economics, Volume 2: Quantum Computing Physics, Blockchains, and Deep Learning Smart Networks, World Scientific 2020 (doi:10.1142/q0243)

• Stefano Mangini, Francesco Tacchino, Dario Gerace, Daniele Bajoni, Chiara Macchiavello, Quantum computing models for artificial neural networks, EPL (Europhysics Letters) 134(1), 10002 (2021) (arXiv:2102.03879)

review:

• Kamila Zaman, Alberto Marchisio, Muhammad Abdullah Hanif, Muhammad Shafique, A Survey on Quantum Machine Learning: Current Trends, Challenges, Opportunities, and the Road Ahead [arXiv:2310.10315]

and with emphasis on classically controlled NISQ-computes:

• John Preskill, Section 6.5 of: Quantum Computing in the NISQ era and beyond, Quantum 2018-08-06, volume 2, page 79 (arXiv:1801.00862, doi:10.22331/q-2018-08-06-79)

• Kosuke Mitarai, Makoto Negoro, Masahiro Kitagawa, Keisuke Fujii, Quantum Circuit Learning, Phys. Rev. A 98, 032309 (2018) (arXiv:1803.00745)

• Marcello Benedetti, Erika Lloyd, Stefan Sack, Mattia Fiorentini, Parameterized quantum circuits as machine learning models, Quantum Science and Technology 4, 043001 (2019) (arXiv:1906.07682)

• Marco Radic, Quantum-enhanced Machine Learning in the NISQ era, 2019 (pdf, pdf)

• Dario Gerace, Quantum Machine Learning on NISQ hardware,, talk at INT online program on “Scientific Quantum Computing and Simulation on Near-Term Devices” 2020 (pdf, pdf)

• Andrea Mari, Thomas R. Bromley, Josh Izaac, Maria Schuld, Nathan Killoran, Transfer learning in hybrid classical-quantum neural networks, Quantum 4, 340 (2020) (arXiv:1912.08278)

• Lucas Munds, Quantum Machine Learning: A Roadmap for NISQ Era and Beyond, Oct 17, 2020

• Tirthajyoti Sarkar, TensorFlow Quantum: Marrying machine learning with quantum computing, Mar 11, 2020

• QunaSys, Tech Blog, Quantum machine learning of quantum data with NISQ devices (Feb 15 2021)

• TensorFlow.org (Google), Quantum Machine Learning

Critical discussion:

• Pablo Bermejo, Paolo Braccia, Manuel S. Rudolph, Zoë Holmes, Lukasz Cincio, M. Cerezo: Quantum Convolutional Neural Networks are (Effectively) Classically Simulable [arXiv:2408.12739]

  “We present this as a challenge to the community and argue that until the viability of QCNNs for [not classically simulable] datasets can be demonstrated there is no reason to believe QCNNs will be useful. […] Hence we boldly claim: There is currently no evidence that QCNNs will work on classically non-trivial tasks, and their place in the upper echelon of promising QML architectures should be seriously revised.”

Applications

There are or will be innumerable applications. Here are some:

To mathematical structures in algebraic geometry, representation theory, number theory and combinatorics:

• Yang-Hui He, Machine-Learning Mathematical Structures, International Journal of Data Science in the Mathematical Sciences [arXiv:2101.06317, doi:10.1142/S2810939222500010]

reviewed in:

• Yang-Hui He, Machine-Learning Mathematical Structures, talk at M-Theory and Mathematics 2023, NYU Abu Dhabi (Jan 2023) [web]

To the conformal bootstrap:

• Gergely Kántor, Vasilis Niarchos, Constantinos Papageorgakis, Solving conformal field theories with artificial intelligence (arXiv:2108.08859)

(…)

Coupling proof assistants with machine learning

On coupling proof assistants with machine learning:

• Li-An Yang et al, Automatically Proving Mathematical Theorems with Evolutionary Algorithms and Proof Assistants [arXiv:1602.07455]

• Klaus Mainzer: From Proof Theory to Proof Assistants – Challenges of Responsible Software and AI, talk at Arbeitstagung Bern-München (2019) [pdf]

• Kaiyu Yang, Jia Deng: Learning to Prove Theorems via Interacting with Proof Assistants [arXiv:1905.09381]

• Cezary Kaliszyk: Machine Learning for Theorem Proving, lecture notes (2023) [pdf:1, 2, 3, 4, 5, 6]

• Garrett Mills: Adventures in AI-assisted proof generation (Mar 2023)

• NeurIPS 2023: Tutorial on Machine Learning for Theorem Proving (2023)

• Terence Tao: Machine-Assisted Proof, Notices of the AMS 72 1 (2024) [doi:10.1090/noti3041, pdf, full issue:pdf]

• Terence Tao: Machine-Assisted Proof, AMS Colloquium Lectures at the 2024 Joint Mathematics Meetings [video:YT]

• Peiyang Song, Kaiyu Yang, Anima Anandkumar: Towards Large Language Models as Copilots for Theorem Proving in Lean [arXiv:2404.12534]

• Leni Aniva et al.: Pantograph: A Machine-to-Machine Interaction Interface for Advanced Theorem Proving, High Level Reasoning, and Data Extraction in Lean 4 [arXiv:2410.16429]

• Kaiyu Yang et al.: Formal Mathematical Reasoning: A New Frontier in AI [arXiv:2412.16075]