Head, Division of Mathematical Sciences. Associate Chair (Faculty), School of Physical and Mathematical Sciences. Nanyang Technological University, Singapore

- Learning of Dynamic Processes -

Abstract: The last decade has seen the emergence of learning techniques that use the computational power of dynamical systems for information processing. Some of those paradigms are based on architectures that are partially randomly generated and require a relatively cheap training effort, which makes them ideal in many applications. The need for a mathematical understanding of the working principles underlying this approach, collectively known as Reservoir Computing, has led to the construction of new techniques that put together well-known results in systems theory and dynamics with others coming from approximation and statistical learning theory. This combination has allowed in recent times to elevate Reservoir Computing to the realm of provable machine learning paradigms and, as we will see in this talk, it also hints at various connections with kernel maps, structure-preserving algorithms, and physics-inspired learning.

References:

• Gonon, L., Grigoryeva, L., and Ortega, J.-P. [2022] Approximation bounds for random neural networks and reservoir systems. To appear in The Annals of Applied Probability. Paper

• Cuchiero, C., Gonon, L., Grigoryeva, L., Ortega, J.-P., and Teichmann, J. [2021] Expressive power of randomized signature. NeurIPS 2021. Paper

• Cuchiero, C., Gonon, L., Grigoryeva, L., Ortega, J.-P., and Teichmann, J. [2021] Discrete-time signatures and randomness in reservoir computing. IEEE Transactions on Neural Networks and Learning Systems, 33(11):6321-6330. Paper

• Gonon, L. and Ortega, J.-P. [2021] Fading memory echo state networks are universal. Neural Networks, 138, 10-13. Paper

• Gonon, L., Grigoryeva, L., and Ortega, J.-P. [2020] Risk bounds for reservoir computing. Journal of Machine Learning Research, 21(240), 1-61. Paper

• Gonon, L. and Ortega, J.-P. [2020] Reservoir computing universality with stochastic inputs. IEEE Transactions on Neural Networks and Learning Systems, 31(1), 100-112. Paper

• Grigoryeva, L. and Ortega, J.-P. [2019] Differentiable reservoir computing. Journal of Machine Learning Research, 20(179), 1-62. Paper

• Grigoryeva, L. and Ortega, J.-P. [2018] Echo state networks are universal. Neural Networks, 108, 495-508. Paper
• Grigoryeva, L. and Ortega, J.-P. [2018] Universal discrete-time reservoir computers with stochastic inputs and linear readouts using non-homogeneous state-affine systems. Journal of Machine Learning Research, 19(24), 1-40. Paper