Title: On Privacy and Personalization in Federated Learning: Analyses and Applications

Abstract:

Recent advances in machine learning often rely on large and centralized datasets. However, curating such data can be challenging when they hold private information, and policies/regulations may mandate that they remain distributed across data silos (e.g. mobile devices or hospitals). Federated learning (FL) has emerged as a paradigm for learning from such distributed data, though it has been shown that its data minimization principle alone may not provide adequate privacy protection. To this end, past work has applied differential privacy (DP) to various parts of the FL pipeline to obtain formal privacy guarantees.

In this talk, we study the application of differential privacy in cross-silo federated learning, a setting characterized by a limited number of resource-abundant clients each with many data subjects. We examine a natural privacy granularity for such settings and reconsider the role of model personalization and its interplay with privacy and statistical heterogeneity. We also establish mean-regularized multi-task learning as a simple and strong baseline, providing an empirical and theoretical characterization of its behaviors. Finally, we describe how these insights helped us develop a winning solution at the US/UK PETs prize challenge.

Committee:

Prof. Artur Dubrawski (co-advisor)

Prof. Virginia Smith (co-advisor)

Prof. Steven Wu

Prof. Elaine Shi

Shengyuan Hu