The asymptotic structure of deep neural networks

Abstract

Deep Neural Networks (DNNs) are the main concept at the center of the artificial intelligence revolution we are experiencing. However, some of the reasons behind their effectiveness (for instance, why do they seem to provide ``good’’ solutions, determined by simple optimization algorithms?), as well as the causes of their limitations (for instance, why are they so parameter and data expensive?), remain somewhat unclear. Therefore, a theoretical/mathematical clarification of these issues would be welcomed and, in principle, might help us in the construction of a new generation of interpretable, safer, sustainable and, consequently, more reliable AI models. With that in mind, a mathematical approach that has provided some relevant insights is the study of the asymptotic structure of DNNs. In this article, we will start by introducing the basics of DNNs, followed by a presentation of some results concerning the study of the large width limit of these models and a discussion of the implications that such results have in our understanding of supervised machine learning with DNNs.