Adam: A Method for Stochastic Optimization

E182822

"Adam: A Method for Stochastic Optimization" is a highly influential machine learning paper that introduces the Adam optimizer, a widely used adaptive gradient-based optimization algorithm for training deep neural networks.

All labels observed (3)

How this entity was disambiguated

Statements (49)

Predicate Object
instanceOf machine learning paper ⓘ
scientific paper ⓘ
abbreviation ICLR ⓘ
application large-scale machine learning problems ⓘ
training deep neural networks ⓘ
author Diederik P. Kingma ⓘ
Jimmy Ba ⓘ
basedOn stochastic gradient descent ⓘ
comparesWith AdaGrad ⓘ
RMSProp ⓘ
SGD with momentum ⓘ
defaultSetting beta1 = 0.9 ⓘ
beta2 = 0.999 ⓘ
epsilon = 1e-8 ⓘ
field deep learning ⓘ
machine learning ⓘ
optimization ⓘ
hyperparameter beta1 ⓘ
beta2 ⓘ
epsilon ⓘ
learning rate ⓘ
impact widely used optimizer in deep learning frameworks ⓘ
influenced AdamW optimizer ⓘ
linked to: Adam optimizer

variants of adaptive gradient methods ⓘ
introduces Adam optimizer ⓘ
introducesConcept bias-corrected first moment estimate ⓘ
bias-corrected second moment estimate ⓘ
licenseOfImplementation open-source implementations available ⓘ
propertyClaimed computationally efficient ⓘ
invariant to diagonal rescaling of gradients ⓘ
little memory requirement ⓘ
well suited for problems with large data ⓘ
well suited for problems with large parameters ⓘ
proposes adaptive moment estimation ⓘ
shortTitle Adam ⓘ
status highly cited ⓘ
title Adam: A Method for Stochastic Optimization ⓘ
topic adaptive learning rate methods ⓘ
gradient-based optimization ⓘ
stochastic optimization ⓘ
usedIn Keras ⓘ
PyTorch ⓘ
TensorFlow ⓘ
uses exponentially decaying averages of past gradients ⓘ
exponentially decaying averages of squared gradients ⓘ
first moment estimates of gradients ⓘ
second moment estimates of gradients ⓘ
venue International Conference on Learning Representations ⓘ
linked to: ICLR
yearProposed 2014 ⓘ

How these facts were elicited

Referenced by (13)

Full triples — surface form annotated when it differs from this entity's canonical label.

Jimmy Ba → coAuthorOf → Adam: A Method for Stochastic Optimization ⓘ
Jimmy Ba → hasNotableWork → Adam: A Method for Stochastic Optimization ⓘ
Adam optimizer → fullName → Adaptive Moment Estimation ⓘ
linked to: Adam: A Method for Stochastic Optimization
Adam optimizer → introducedInPaper → Adam: A Method for Stochastic Optimization ⓘ
Adam: A Method for Stochastic Optimization → title → Adam: A Method for Stochastic Optimization ⓘ
Adam: A Method for Stochastic Optimization → proposes → adaptive moment estimation ⓘ
linked to: Adam: A Method for Stochastic Optimization
Diederik P. Kingma → coAuthorOf → Adam: A Method for Stochastic Optimization ⓘ
Adam → fullName → Adaptive Moment Estimation ⓘ
linked to: Adam: A Method for Stochastic Optimization
Adam → introducedInPaper → Adam: A Method for Stochastic Optimization ⓘ
Adam → describedIn → Adam: A Method for Stochastic Optimization ⓘ
Adam → abbreviationFor → Adaptive Moment Estimation ⓘ
linked to: Adam: A Method for Stochastic Optimization
Jimmy Lei Ba → coAuthorOf → Adam: A Method for Stochastic Optimization ⓘ
Jimmy Lei Ba → notableWork → Adam: A Method for Stochastic Optimization ⓘ