Abstract:

In this talk, recent results on various aspects of the Empirical Risk Minimization (ERM) problem with Relative Entropy Regularization (ERM-RER) are presented. The regularization is with respect to a sigma-finite measure, instead of a probability measure, which provides a larger flexibility for including prior knowledge on the models. Special cases of this general formulation include the ERM problem with (discrete or differential) entropy regularization and the information-risk minimization problem. Three results are discussed. First, it is shown that the empirical risk observed when models are sampled from the ERM-RER optimal probability measure is a sub-Gaussian random variable that exhibits a probably-approximately-correct (PAC) guarantee for the ERM problem. Second, the sensitivity of the expected empirical risk to deviations from the ERM-RER-optimal measure is characterized. Finally, using the notion of sensitivity, the impact of data aggregation on the generalization capabilities of machine learning algorithms based on the ERM-RER is studied. Interestingly, none of these results relies on statistical assumptions on the datasets, and thus, cases in which  datasets exhibit different statistical properties can also be studied within this framework. 

Bio:

Samir M. Perlaza is a permanent member of the scientific staff at INRIA, the French Institute for Research in Computer Science and Applied Mathematics; an associate member of the GAATI Laboratory of the University of French Polynesia; and a visiting research collaborator in the Department of Electrical and Computer Engineering at Princeton University. He received the M.Sc. and Ph.D. degrees from École Nationale Supérieure des Télécommunications (Telecom ParisTech) in 2008 and 2011, respectively. From 2008 to 2011, he was also a research engineer at France Télécom - Orange Labs (Paris, France). He has held long-term academic appointments at the Alcatel-Lucent Chair in Flexible Radio at Supélec and as post-doctoral fellow at Princeton. Dr. Perlaza's research interests are in the areas of information theory, game theory, data sciences, and their applications in wireless networks, power systems, and artificial intelligence. Among his publications in these areas is the recent book ‘‘Advanced Data Analytics for Power Systems’’ (Cambridge University Press, 2021) co-authored with Ali Tajer and H. Vincent Poor. Dr. Perlaza has served as an Editor of the IEEE TRANSACTIONS ON COMMUNICATIONS and the IET Smart Grid Journal. Recognition of his work includes the Alban Fellowship and the Marie Sklodowska-Curie Fellowship, both from the European Commission.

Seminar jointly sponsored by the Center for Statistics and Machine Learning and the Department of Electrical and Computer Engineering.