Learning tasks involving function approximation are preva- lent in numerous domains of science and engineering. The underlying idea is to design a learning algorithm that gener- ates a sequence of functions converging to the desired target function with arbitrary accuracy by using the available data samples. In this paper, we present a novel interpretation of iterative function learning through the lens of ensemble dy- namical systems, with an emphasis on establishing the equiv- alence between convergence of function learning algorithms and asymptotic behavior of ensemble systems. In particular, given a set of observation data in a function learning task, we prove that the procedure of generating an approximation sequence can be represented as a steering problem of a dy- namic ensemble system defined on a function space. This in turn gives rise to an ensemble systems-theoretic approach to the design of “continuous-time” function learning algorithms, which have a great potential to reach better generalizability compared with classical “discrete-time” learning algorithms.
Preprint version When Machine Learning meets Dynamical Systems: Theory and Applications (MLmDS), AAAI 2023, 2023.
Copyright © 2023, Association for the Advancement of Artificial Intelligence (www.aaai.org), AAAI 2023 Workshop “When Machine Learning meets Dynamical Systems: Theory and Applications” (MLmDS 2023). All rights reserved.
Zhang, W., Narayanan, V., & Li, J. (2023). Dynamic function learning through control of ensemble systems [Preprint]. When Machine Learning meets Dynamical Systems: Theory and Applications (MLmDS), AAAI 2023, Washington, DC.