A Simple Modified Frank-Wolfe Method for sparse constrained optimization
Accelerating Frank-Wolfe via Averaging Step Directions
We consider a modified version of the frank-wolfe method for sparse constrained optimization, where the step direction is a simple weighted average of past oracle calls.This method requires very little memory and computation overhead, and provably decays this discretization error term.We show that this method improves the convergence rate over several problems, especially after the sparse manifold has been detected.Theoretically, we show the method has an overall global convergence rate of where after manifold identification, this rate speeds to.We also observe that the method achieves this accelerated rate from a very early stage, suggesting a promising mode of acceleration for this family of methods.
