A Zeroth-Order Accelerated Zeroth-Order Momentum Method for Non-Max and White-Box Minimax-optimization Problems

Home

Discover

Bookmarks

Keep Up With Top Trending Papers In CS / AI / ML ...

Source arXiv

Computer Vision

Machine Learning

Optimization and Control

A Zeroth-Order Accelerated Zeroth-Order Momentum Method for Non-Max and White-Box Minimax-optimization Problems

Accelerated Zeroth-Order Momentum Methods from Mini to Minimax Optimization

In the paper, we propose a new accelerated zeroth-order momentum (acc-zom)method to solve the non-convex stochastic mini-optimization problems and propose an accelerated zeroth-order momentum descent ascent (acc-mda)method for solving the white-box minimax problems.

We prove that the acc-zom method achieves a lower query complexity of for finding an point, which improves the best known result by a factor of where denotes the parameter dimension.

In particular, our theoretical result does not rely on large batchesrequired in the existing methods.

Further, we extend the acc-zom method to solve the non-convex stochastic stochasticminimax-optimization problems and propose an accelerated zeroth-order momentum descent ascent (acc-zomda)method.

We prove that the acc-zomda method reaches the best know query complexity of for finding an point, where and denote dimensions of the mini and max optimization parameters respectively and is condition number.

Extensive experimental results on the black-box adversarial attack to deep neural networks (dnns) and poisoning attack demonstrate the efficiency of our algorithms.

Authors

Feihu Huang, Shangqian Gao, Jian Pei, Heng Huang

◐ Latest Activity

◐ Recommended Members