Zoran Skoda machine learning

General

• Ian Goodfellow Y. Bengio, A. Courville, Deep learning, pdf MIT Press 2016
• wikipedia: supervised learning, statistical classification, regression analysis, backpropagation, cluster analysis, dimensionality reduction. structured prediction, anomaly detection, artificial neural network, reinforcement learning, glossary of artificial intelligence, statistical semantics
• $n$Lab:machine learning, singular learning theory, neural network, kernel method, topological data analysis, topological machine learning, statistical learning theory, information geometry
• singular learning theory seminar page
• Juergen Schmidhuber (2015), Scholarpedia, 10(11):32832 Deep learning (with extensive historical bibliography)
• azimuth project: deep learning, artificial intelligence
• Journal of Machine Learning Research (open access and also has a depository of free software)
• Godfried’s list of pattern recognition links (some links disfunctional)

Software platforms

• see also cslinks
• mlpack machine learning library for C++
• The Theano development team, Theano: A Python framework for fast computation of mathematical expressions, arxiv/1605.02688
• M. Abadi et al. TensorFlow: Large-scale machine learning on heterogeneous distributed systems, arxiv/1603.04467
• Torch wikipedia
• pytorch at wikipedia, github, pytorch.org, pytorch.org/tutorials
• pandas documentation at pandas-ml

Early references

Pioneering research:

• M. L. Minsky, S. A. Papert, Perceptrons, Cambridge, MA: MIT Press, 1969
• B. E. Boser, I. M. Guyon, V. N. Vapnik, A training algorithm for optimal margin classifiers, In: In D. Haussler, editor, 5th Annual ACM Workshop on COLT, 144-152, ACM Press. (1992)

Learning and teaching theory

Note: part of the bibliography in this subsection is adapted from CS/ECE 861 course webpage from University of Wisconsin, which is created by Jerry Zhu.

• Xiaojin Zhu, Adish Singla, Sandra Zilles, Anna N. Rafferty, An overview of machine teaching, arxiv/1801.05927
• SA Goldman, MJ Kearns, On the complexity of teaching, pdf 1995
• Steve Hanneke, Theory of active learning, pdf
• Shai Shalev-Shwartz, Shai Ben-David, Understanding machine learning: from theory to algorithms, webpage Cambridge University Press 2014
• Roman Vershynin, High-dimensional probability: an introduction with applications in data science, bookdraft, pdf
• Andreas Krause, Daniel Golovin, Submodular function maximization, survey, pdf
• Shai Shalev-Shwartz, Online learning and online convex optimization, Foundations and trends in machine learning, 4:2, 107-194 (2011) doi pdf
• Sébastien Bubeck, Nicolò Cesa-Bianchi, Regret analysis of stochastic and nonstochastic multi-armed bandit problems, Foundations and trends in machine learning 5:1 (2012) 1–122 pdf arxiv/1204.5721 doi
• А.Я. Червоненкис, Теория обучения машин, Курс лекций в Школе Анализа Данных (Яндекс-школе), 22 лекции по методам машинного обучения и распознаванию образов
• В. Н. Вапник, А. Я. Червоненкис, Теория равномерной сходимости частот появления событий к их вероятностям и задачи поиска оптимального решения по эмпирическим данным // Автоматика и телемеханика. — 1971. — Т. 32, № 2. — С. 207—317. — ISSN 0005-2310 pdf
• Вапник В. Н., Червоненкис А. Я. Теория распознавания образов. М.: Наука, 1974
• V. Vapnik, An overview of statistical learning theory - Neural Networks, IEEE trans. on neural networks, 10:5 (1999) pdf
• V. N. Vapnik, Statistical Learning Theory, 2nd. ed. John-Wiley and Sons. New York, 1998
• S. Amari, A theory of adaptive pattern classifiers, IEEE Transactions on Electronic Computers, EC-16(3), 299–307 doi
• Matus Telgarsky, Benefits of depth in neural networks, arxiv.org/1602.04485
• F. Cucker, S. Smale, On the mathematical foundations of learning, Bull. AMS 39:1, 1–49, 2002 pdf mirror
• S. Ben-David, P. Hrubes, S. Moran, A. Yehudayoff, Learnability can be undecidable, Nature Machine intelligence, 1 (2019) 44-48.

Artificial general intelligence

The following framework claims to relate to homotopy type theory?

• Ben Goertzel, Reflective metagraph rewriting as a foundation for an AGI “Language of Thought”, arXiv:2112.08272

On a definition of artificial general intelligence

• S. Legg, M. Hutter, Universal intelligence: a definition of machine intelligence, Minds & Machines 17, 391–444 (2007) doi

• M. Hutter, Universal artificial intelligence: sequential decisions based on algorithmic probability, Springer 2005; book presentation pdf

• Shane Legg, Machine super intelligence, PhD thesis, 2008 pdf

Categories, (tropical, information etc.) geometry versus machine learning

• Jared Culbertson, Kirk Sturtz, Bayesian machine learning via category theory, arxiv/1312.1445
• Liwen Zhang, Gregory Naitzat, Lek-Heng Lim, Tropical geometry of deep neural networks, arxiv/1805.07091
• Shun-ichi Amari, Ryo Karakida, Masafumi Oizumi, Fisher information and natural gradient learning of random deep networks, arxiv/1808.07172

Other

Some notable people and institutions

• Max Planck Institute for Intelligent Systems and its Empirical Inference Department, Tuebingen
• Vladimir Vapnik, wikipedia

General and conceptual

• Chuanqi Tan, Fuchun Sun, Tao Kong, Wenchang Zhang, Chao Yang, Chunfang Liu, A survey on deep transfer learning, arxiv/1808.01974 (a way to address insufficient data in some cases)

• Joachim Schmidthuber, Deep learning in neural networks:an overview arxiv/1404.7828; cf. also short html version with some add. links and the beautiful historical insights in his short article Who invented backpropagation? (a g+ version here) and in his scholarpedia Deep learning contribution

• David E. Rumelhart, Geoffrey E. Hinton & Ronald J. Williams, Learning representations by back-propagating errors, Nature 323, 533–536 (1986) pdf doi

• Hinton, Srivastava, Swersky, Coursera course, Neural networks for machine learning videos

• William Gilpin, Cellular automata as convolutional neural networks, arxiv/1809.02942

• Eunji Jeong, Joo Seong Jeong, Soojeong Kim, Gyeong-In Yu, Byung-Gon Chun, Improving the expressiveness of deep learning frameworks with recursion, arxiv/1809.00832

• Dorjan Hitaj, Luigi V. Mancini, Have you stolen my model? Evasion attacks against deep neural network watermarking techniques, arxiv/1809.00615

• Nicholas Polson, Vadim Sokolov, Deep learning: computational aspects, arxiv /1808.08618

• Alessandro Betti, Marco Gori, Giuseppe Marra, Backpropagation and biological plausibility, arxiv/1808.06934

• Gamaleldin F. Elsayed, Ian Goodfellow, Jascha Sohl-Dickstein, Adversarial reprogramming of neural networks, arxiv/1806.11146; Tom B. Brown etc. and Ian Goodfellow, Unrestricted adversarial examples arxiv/1809.08352

• Sanjoy Dasgupta, Charles F. Stevens, Saket Navlakha, A neural algorithm for a fundamental computing problem (motivated by fruit fly biological neural network) Science 10 Nov 2017: 358:6364, pp. 793-796 doi

• Yves Chauvin, David E. Rumelhart (editors) Backpropagation: Theory, Architectures, and Applications, 576 pp. in series Developments in Connectionist Theory, Lawrence Erlbaum Associates 1995

• Daniel Crespin, Generalized backpropagation (written for mathematicians/diff.geom.) pdf

• FFX shallow algorithm

• Jean-Baptiste Bardin, Gard Spreemann, Kathryn Hess, Topological exploration of artificial neuronal network dynamics, arxiv/1810.01747

• Tomasz Maszczyk, Włodzisław Duch, Support Feature Mahcines: support vectors are not enough, arxiv/1901.09643

• Pankaj Mehta, David J. Schwab, An exact mapping between the variational renormalization group and deep earning, arxiv/1410.3831

• A. Jakovac, D. Berenyi, P. Posfay, Understanding understanding: a renormalization group inspired model of (artificial) intelligence, arxiv/2010.13482

• M. M. Bronstein et al. Geometric deep learning: going beyond Euclidean data, IEEE Signal Processing Magazine 34:4 (2017) arxiv/1611.08097 doi

With specific kinds of applications

• Emmanuel de Bezenac, Arthur Pajot, Patrick Gallinari, Deep learning for physical processes: incorporating prior scientific knowledge, arxiv/1711.07970
• Ryan Ferguson, Andrew Green, Applying deep learning to derivatives valuation, arxiv/1809.02233
• Yurui Qu, Li Jing, Yichen Shen, Min Qiu, Marin Soljacic, Migrating knowledge between physical scenarios based on artificial neural networks, https://arxiv/1809.00972; Li Jing et al. Tunable efficient unitary neural networks (EUNN) and their application to RNNs, https://arxiv/1612.05231
• Lorenzo Pastori, Raphael Kaubruegger, Jan Carl Budich, Generalized transfer matrix states from artificial neural networks, arxiv/1808.02069 (wavefunctions)
• Jaideep Pathak, Alexander Wikner, Rebeckah Fussell, Sarthak Chandra, Brian Hunt, Michelle Girvan, Edward Ott, Hybrid forecasting of chaotic processes: using machine learning in conjunction with a knowledge-based model, arxiv/1803.04779 and related popular article in quantamagazine
• Abigail See, Four deep learning trends from ACL2017 (comprehensive conference report in blog form)

Some popular articles

• https://www.theatlantic.com/technology/archive/2018/05/machine-learning-is-stuck-on-asking-why/560675