Teacher Development Center, Human Resources Department‘s "Distinguished Scholar Lecture" activity has invited Professor Zhouchen Lin of Peking University for an academic exchange. Below are the details of the session, students and faculty members are welcome to attend.
Topic:Optimization Induced Equilibrium Networks: An Explicit Optimization Perspective for Understanding Equilibrium Models
Speaker: ProfessorZhouchen Lin(IEEE Fellow, Peking University)
Time: 14:00 (Saturday), August 20, 2022
Venue: Qingshuihe Campus, Coffee beanery
Host: Professor Chunming Li, School of Information and Communication Engineering
Introduction:
To reveal the mystery behind deep neural networks (DNNs), optimization may offer a good perspective. There are already some clues showing the strong connection between DNNs and optimization problems, e.g., under a mild condition, DNN’s activation function is indeed a proximal operator. In this paper, we are committed to providing a unified optimization induced interpretability for a special class of networks—equilibrium models, i.e., neural networks defined by fixed point equations, which have become increasingly attractive recently. To this end, we first decompose DNNs into a new class of unit layer that is the proximal operator of an implicit convex function while keeping its output unchanged. Then, the equilibrium model of the unit layer can be derived, we name it Optimization Induced Equilibrium Networks (OptEq). The equilibrium point of OptEq can be theoretically connected to the solution of a convex optimization problem with explicit objectives. Based on this, we can flexibly introduce prior properties to the equilibrium points: 1) modifying the underlying convex problems explicitly so as to change the architectures of OptEq; and 2) merging the information into the fixed point iteration, which guarantees to choose the desired equilibrium point when the fixed point set is non-singleton. We show that OptEq outperforms previous implicit models even with fewer parameters.
Speakers’ profiles：
Zhouchen Lin received the Ph.D. degree in applied mathematics from Peking University in 2000. He is currently a Professor with the Key Laboratory of Machine Perception (MOE), School of Artificial Intelligence, Peking University. He is a Fellow of the IAPR, the IEEE, and the CSIG. He is also a recepient of The National Science Fund for Distinguished Young Scholars. His research interests include machine learning and numerical optimization. He has published over 260 technical papers and 4 monographs, receiving over 25,000 Google Scholar citations. He has been Area Chairs of ACML, ACCV, CVPR, ICCV, NIPS/NeurIPS, AAAI, IJCAI, ICLR, and ICML for many times. He is currently a Program co-Chair of ICPR 2022 and Senior Area Chairs of ICML 2022, NeurIPS 2022, and CVPR 2023. He was an Associate Editor of the IEEE Transactions on Pattern Analysis and Machine Intelligence and currently is an associate editor of the International Journal of Computer Vision and Optimization Methods and Software.
Organizer：Teacher Development Center, Human Resources Department
Co-organizer：School of Information and Communication Engineering
