Keep Up With Latest Trending Papers. Computer Science, AI and Machine Learning and more.Subscribe

Top Papers in Gradient descent algorithm

Share

Revisiting Quantum Linear System Solving from the Perspective of convex optimization and gradient descent-type algorithms

Improving quantum linear system solvers via a gradient descent perspective

Read More...

Share

Learning the Principal Subspace of a Matrix from Sample entries

A Novel Stochastic Gradient Descent Algorithm for Learning Principal Subspaces

Read More...

Share

An Algebraically Converging Stochastic Gradient Descent Algorithm for Global Optimization

We propose a new stochastic gradient descent algorithm for finding the global
optimizer of nonconvex optimization problems, referred to here as "AdaVar". A
key component in the algorithm is the adapti

More...

Share

Quantum gradient descent algorithms for nonequilibrium steady states and linear algebraic systems

The gradient descent approach is the key ingredient in variational quantum
algorithms and machine learning tasks, which is an optimization algorithm for
finding a local minimum of an objective functio

More...

Share

Improving Computational Complexity in Statistical Models with Second-Order Information

It is known that when the statistical models are singular, i.e., the Fisher
information matrix at the true parameter is degenerate, the fixed step-size
gradient descent algorithm takes polynomial numb

More...

Share

Optimal control for interacting particle systems driven by neural networks

We propose a neural network approach to model general interaction dynamics
and an adjoint based stochastic gradient descent algorithm to calibrate its
parameters. The parameter calibration problem is

More...

Share

Towards Statistical and Computational Complexities of Polyak Step Size Gradient Descent

We study the statistical and computational complexities of the Polyak step
size gradient descent algorithm under generalized smoothness and Lojasiewicz
conditions of the population loss function, name

More...

Share

Stochastic gradient descent on Riemannian manifolds

Stochastic gradient descent is a simple approach to find the local minima of
a cost function whose evaluations are corrupted by noise. In this paper, we
develop a procedure extending stochastic gradie

More...

Share

Accelerating ptychographic reconstructions using spectral initializations

Ptychography is a promising phase retrieval technique for label-free
quantitative phase imaging. Recent advances in phase retrieval algorithms
witnessed the development of spectral methods, in order t

More...

Share

The Hamilton-Jacobi Theory of Continuous-Time Markov Processes

Action Functional Gradient Descent algorithm for estimating escape paths in Stochastic Chemical Reaction Networks

Read More...

Share

Riemannian optimization for non-centered mixture of scaled Gaussian distributions

This paper studies the statistical model of the non-centered mixture of
scaled Gaussian distributions (NC-MSG). Using the Fisher-Rao information
geometry associated to this distribution, we derive a R

More...

Share

The Information Geometry of Mirror Descent

Information geometry applies concepts in differential geometry to probability
and statistics and is especially useful for parameter estimation in exponential
families where parameters are known to lie

More...

Share

Machine Learning

Computer Vision

Optimization and Control

Artificial Intelligence

Stats Machine Learning

Gradient Descent Finds Global Minima of Deep Neural Networks

Gradient descent finds a global minimum in training deep neural networks
despite the objective function being non-convex. The current paper proves
gradient descent achieves zero training loss in polyn

More...

Share

Uniform Abductive and Function Properties of Gradient Descent

On Uniform Boundedness Properties of SGD and its Momentum Variants

Read More...

Share

Decentralized Stochastic Optimization with Inherent Privacy Protection

Decentralized stochastic optimization is the basic building block of modern
collaborative machine learning, distributed estimation and control, and
large-scale sensing. Since involved data usually con

More...

Share

A Multi-parameter Updating Fourier Online Gradient Descent Algorithm for Large-scale Nonlinear Classification

Large scale nonlinear classification is a challenging task in the field of
support vector machine. Online random Fourier feature map algorithms are very
important methods for dealing with large scale

More...

Share

Nesterov's method with decreasing learning rate leads to accelerated stochastic gradient descent

We present a coupled system of ODEs which, when discretized with a constant
time step/learning rate, recovers Nesterov's accelerated gradient descent
algorithm. The same ODEs, when discretized with a

More...

Share

The convergence properties of KSD Descent

Kernel Stein Discrepancy Descent

Read More...

Share

Provable Phase Retrieval with Mirror Descent

In this paper, we consider the problem of phase retrieval, which consists of
recovering an $n$-dimensional real vector from the magnitude of its $m$ linear
measurements. We propose a mirror descent (o

More...

Share

Alternating Mirror Descent for Constrained Zero-Sum Games

Alternating Mirror Descent for Constrained Min-Max Games

Read More...

Share

More

More