Inria, Ecole Normale Supérieure

Title : Information Theory with Kernel Methods

Abstract: Estimating and computing entropies of probability distributions are key computational tasks throughout data science. In many situations, the underlying distributions are only known through the expectation of some feature vectors, which has led to a series of works within kernel methods. In this talk, I will explore the particular situation where the feature vector is a rank-one positive definite matrix, and show how the associated expectations (a covariance matrix) can be used with information divergences from quantum information theory to draw direct links with the classical notions of Shannon entropies.

References:

Francis Bach. Information Theory with Kernel Methods. To appear in IEEE Transactions in Information Theory, 2022. https://arxiv.org/pdf/2202.08545