Adam: A Method for Stochastic Optimization

E182822

"Adam: A Method for Stochastic Optimization" is a highly influential machine learning paper that introduces the Adam optimizer, a widely used adaptive gradient-based optimization algorithm for training deep neural networks.

All labels observed (3)

How this entity was disambiguated

Statements (49)

Predicate Object
instanceOf machine learning paper
scientific paper
abbreviation ICLR
application large-scale machine learning problems
training deep neural networks
author Diederik P. Kingma
Jimmy Ba
basedOn stochastic gradient descent
comparesWith AdaGrad
RMSProp
SGD with momentum
defaultSetting beta1 = 0.9
beta2 = 0.999
epsilon = 1e-8
field deep learning
machine learning
optimization
hyperparameter beta1
beta2
epsilon
learning rate
impact widely used optimizer in deep learning frameworks
influenced AdamW optimizer
linked to: Adam optimizer

variants of adaptive gradient methods
introduces Adam optimizer
introducesConcept bias-corrected first moment estimate
bias-corrected second moment estimate
licenseOfImplementation open-source implementations available
propertyClaimed computationally efficient
invariant to diagonal rescaling of gradients
little memory requirement
well suited for problems with large data
well suited for problems with large parameters
proposes adaptive moment estimation
shortTitle Adam
status highly cited
title Adam: A Method for Stochastic Optimization
topic adaptive learning rate methods
gradient-based optimization
stochastic optimization
usedIn Keras
PyTorch
TensorFlow
uses exponentially decaying averages of past gradients
exponentially decaying averages of squared gradients
first moment estimates of gradients
second moment estimates of gradients
venue International Conference on Learning Representations
linked to: ICLR
yearProposed 2014

How these facts were elicited

Referenced by (7)

Full triples — surface form annotated when it differs from this entity's canonical label.

Jimmy Ba coAuthorOf Adam: A Method for Stochastic Optimization
Jimmy Ba hasNotableWork Adam: A Method for Stochastic Optimization
Adam optimizer fullName Adaptive Moment Estimation
linked to: Adam: A Method for Stochastic Optimization
Adam optimizer introducedInPaper Adam: A Method for Stochastic Optimization
Adam: A Method for Stochastic Optimization title Adam: A Method for Stochastic Optimization
Adam: A Method for Stochastic Optimization proposes adaptive moment estimation
linked to: Adam: A Method for Stochastic Optimization
Diederik P. Kingma coAuthorOf Adam: A Method for Stochastic Optimization