Keep Up With Latest Trending Papers. Computer Science, AI and Machine Learning and more.Subscribe

Top Papers in Gradient descent

Share

Learning to Learn without Gradient Descent by Gradient Descent

We learn recurrent neural network optimizers trained on simple synthetic
functions by gradient descent. We show that these learned optimizers exhibit a
remarkable degree of transfer in that they can b

More...

Share

Learning to learn by gradient descent by gradient descent

The move from hand-designed features to learned features in machine learning
has been wildly successful. In spite of this, optimization algorithms are still
designed by hand. In this paper we show how

More...

Share

Sparse Spiking Gradient Descent

There is an increasing interest in emulating Spiking Neural Networks (SNNs)
on neuromorphic computing devices due to their low energy consumption. Recent
advances have allowed training SNNs to a point

More...

Share

MBGDT:Robust Mini-Batch Gradient Descent

In high dimensions, most machine learning method perform fragile even there
are a little outliers. To address this, we hope to introduce a new method with
the base learner, such as Bayesian regression

More...

Share

Competitive Gradient Descent

We introduce a new algorithm for the numerical computation of Nash equilibria
of competitive two-player games. Our method is a natural generalization of
gradient descent to the two-player setting wher

More...

Share

Adaptive Gradient Descent without Descent

We present a strikingly simple proof that two rules are sufficient to
automate gradient descent: 1) don't increase the stepsize too fast and 2) don't
overstep the local curvature. No need for function

More...

Share

FairGrad: Fairness Aware Gradient Descent

We tackle the problem of group fairness in classification, where the
objective is to learn models that do not unjustly discriminate against
subgroups of the population. Most existing approaches are li

More...

Share

Exponentiated Gradient Meets Gradient Descent

The (stochastic) gradient descent and the multiplicative update method are
probably the most popular algorithms in machine learning. We introduce and
study a new regularization which provides a unific

More...

Share

Continuous vs. Discrete Optimization of Deep Neural Networks

Existing analyses of optimization in deep learning are either continuous,
focusing on (variants of) gradient flow, or discrete, directly treating
(variants of) gradient descent. Gradient flow is amena

More...

Share

Added to collectionTop 100 Papers By Signal Trends

Handbook of Convergence Theorems for (Stochastic) Gradient Methods

This is a handbook of simple proofs of the convergence of gradient and
stochastic gradient descent type methods. We consider functions that are
Lipschitz, smooth, convex, strongly convex, and/or Polya

More...

Share

Scalable Estimation and Inference with Large-scale or Online Survival Data

With the rapid development of data collection and aggregation technologies in many scientific disciplines, it is becoming increasingly ubiquitous to conduct large-scale or online regression to analyze

More...

Share

Gradient Correction beyond Gradient Descent

The great success neural networks have achieved is inseparable from the
application of gradient-descent (GD) algorithms. Based on GD, many variant
algorithms have emerged to improve the GD optimizatio

More...

Share

Neural Variational Gradient Descent

Neural Variational Gradient Descent

Read More...

Share

Polymatrix Competitive Gradient Descent

Many economic games and machine learning approaches can be cast as
competitive optimization problems where multiple agents are minimizing their
respective objective function, which depends on all agen

More...

Share

RSGDA: A Randomized variant of the Epoch Gradient textitDescent Ascent Algorithm

Randomized Stochastic Gradient Descent Ascent

Read More...

Share

Machine Learning

Neural and Evolutionary Computing

Optimization and Control

Probability

Stats Machine Learning

Convergence of gradient descent for deep neural networks

Optimization by gradient descent has been one of main drivers of the "deep
learning revolution". Yet, despite some recent progress for extremely wide
networks, it remains an open problem to understand

More...

Share

Blind Descent: A Prequel to Gradient Descent

We describe an alternative to gradient descent for backpropogation through a neural network, which we call Blind Descent. We believe that Blind Descent can be used to augment backpropagation by using

More...

Share

Implicit Gradient Regularization

Gradient descent can be surprisingly good at optimizing deep neural networks
without overfitting and without explicit regularization. We find that the
discrete steps of gradient descent implicitly reg

More...

Share

Reparameterizing Mirror Descent as Gradient Descent

Most of the recent successful applications of neural networks have been based
on training with gradient descent updates. However, for some small networks,
other mirror descent updates learn provably m

More...

Share

Natural Gradient Descent with Generic Metric Spaces

Efficient Natural Gradient Descent Methods for Large-Scale Optimization Problems

Read More...

Share

More

More