Title: Understanding Neural Network Behavior with the Renormalization Group
Abstract: The growing presence and influence of machine learning in both science and society necessitates a better understanding of these algorithms. Interestingly, many neural network behaviors are reminiscent of physical phenomena, such as the empirically-observed approximate power law dependence of a network’s generalization performance on the training dataset’s size; an example of a so-called neural scaling law. The parallel between neural networks and physical systems suggests that the tools of theoretical physics could be leveraged to develop predictive models of neural network behavior.
In this talk, we will explore how the Wilsonian Renormalization Group (RG) can be used to understand and make analytical predictions for the behavior of overparameterized neural networks, including neural scaling laws on real-world datasets. In addition to being analytically tractable, this approach goes beyond structural analogies between RG and neural networks by providing a natural connection between RG flow and learnable vs. unlearnable feature modes. Studying such flows may improve our understanding of feature learning in deep neural networks, and enable us to identify potential universality classes in these models.
For help, please contact Webmaster.