speaker-photo

Federica Gerace

SISSA, Trieste

Federica has an M.Sc. in physics of complex systems from the international track of Politecnico di Torino. She got her PhD in statistical physics, machine learning and computational neuroscience under the guidance of Riccardo Zecchina in Politecnico di Torino. She did a post-doc in statistical physics and machine learning in the lab of Lenka Zdeborová and Florent Krzakala between the Institut de Physique Théorique (IPhT) in Saclay and the École Polytechnique Fédérale de Lausanne (EPFL). She currently works as senior post-doctoral researcher in Theoretical and Scientific Data Science group of Scuola Superiore di Studi Avanzati (SISSA) in Trieste and as co-founder and Artificial Intelligence director in Syndiag s.r.l. ABSTRACT: Transfer learning can significantly improve the sample efficiency of neural networks, by exploiting the relatedness between a data-scarce target task and a data-abundant source task. Despite years of successful applications, transfer learning practice often relies on ad-hoc solutions, while theoretical understanding of these procedures is still limited. In the present work, we re-think a solvable model of synthetic data as a framework for modeling correlation between data-sets. This setup allows for an analytic characterization of the generalization performance obtained when transferring the learned feature map from the source to the target task. Focusing on the problem of training two-layer networks in a binary classification setting, we show that our model can capture a range of salient features of transfer learning with real data. Moreover, by exploiting parametric control over the correlation between the two data-sets, we systematically investigate under which conditions the transfer of features is beneficial for generalization.

15:00-15:30

Thursday April 20th

Probing transfer learning with a model of synthetic correlated datasets

Transfer learning can significantly improve the sample efficiency of neural networks, by exploiting the relatedness between a data-scarce target task and a data-abundant source task. Despite years of successful applications, transfer learning practice often relies on ad-hoc solutions, while theoretical understanding of these procedures is still limited. In the present work, we re-think a solvable model of synthetic data as a framework for modeling correlation between data-sets. This setup allows for an analytic characterization of the generalization performance obtained when transferring the learned feature map from the source to the target task. Focusing on the problem of training two-layer networks in a binary classification setting, we show that our model can capture a range of salient features of transfer learning with real data. Moreover, by exploiting parametric control over the correlation between the two data-sets, we systematically investigate under which conditions the transfer of features is beneficial for generalization.