Adam

E701497

Adam is a widely used stochastic optimization algorithm in machine learning that combines ideas from momentum and adaptive learning rates to efficiently train deep neural networks.

All labels observed (1)

Label Occurrences
Adam canonical 6

How this entity was disambiguated

Statements (46)

Predicate Object
instanceOf optimization algorithm ⓘ
stochastic optimization method ⓘ
abbreviationFor Adaptive Moment Estimation ⓘ
appliedIn computer vision ⓘ
natural language processing ⓘ
reinforcement learning ⓘ
speech recognition ⓘ
basedOn adaptive learning rates ⓘ
momentum ⓘ
stochastic gradient descent ⓘ
commonVariant AMSGrad ⓘ
AdamW ⓘ
linked to: Adam optimizer
comparedWith AdaGrad ⓘ
RMSProp ⓘ
SGD with momentum ⓘ
defaultHyperparameter beta1 = 0.9 ⓘ
beta2 = 0.999 ⓘ
epsilon = 1e-8 ⓘ
learning rate = 0.001 ⓘ
describedIn Adam: A Method for Stochastic Optimization ⓘ
field deep learning ⓘ
machine learning ⓘ
hasProperty computationally efficient ⓘ
handles sparse gradients ⓘ
memory efficient ⓘ
scale invariant to gradient magnitudes ⓘ
suitable for high-dimensional parameter spaces ⓘ
suitable for large datasets ⓘ
implementedIn JAX ⓘ
Keras ⓘ
PyTorch ⓘ
TensorFlow ⓘ
introducedIn 2014 ⓘ
optimizationType first-order method ⓘ
performs bias correction of moment estimates ⓘ
proposedBy Diederik P. Kingma ⓘ
Jimmy Ba ⓘ
publishedAt International Conference on Learning Representations ⓘ
linked to: ICLR
publishedIn 2015 ⓘ
updates parameters with element-wise adaptive learning rates ⓘ
usedFor stochastic optimization ⓘ
training deep neural networks ⓘ
uses exponentially decaying averages of past gradients ⓘ
exponentially decaying averages of past squared gradients ⓘ
first moment estimates of gradients ⓘ
second moment estimates of gradients ⓘ

How these facts were elicited

Referenced by (6)

Full triples — surface form annotated when it differs from this entity's canonical label.