Over-Parametrization in Deep Learning

Home

Discover

Bookmarks

Keep Up With Latest Trending Papers. Computer Science, AI and Machine Learning and more.Subscribe

Liked byLeo

CC BY

Source arXiv

Machine Learning

Stats Machine Learning

Over-Parametrization in Deep Learning

A Universal Law of Robustness via Isoperimetry

We show that for a broad class of data distributions and model classes, overparametrization is necessary if one wants to interpolate the data smoothly.

Namely we show that smooth interpolation requires times more parameters than mere interpolation, where is the ambient data dimension.

We prove this universal law of robustness for any smoothly parametrized function class with polynomial size weights, and any covariate distribution verifying isoperimetry.

We also give an interpretation of our result as an improved generalization bound for modelclasses consisting of smooth functions.

Authors

Sébastien Bubeck, Mark Sellke

◐ Recommended Follows

◐ Latest Activity